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Summary 



Getting the evidence for evidence- 
based initiatives: how the Midwest 
states use data systems to improve 
education processes and outcomes 



States in the Midwest Region are develop- 
ing innovative approaches to collecting 
and providing access to high-quality data 
in order to improve educational decision- 
making. Additional capacity-building and 
increased technical assistance at the state 
and local levels would enhance this work. 

Educational improvement through data-based 
decisionmaking using high-quality data is 
a longstanding goal of policymakers and 
practitioners, and ensuring the quality of the 
evidence available to inform such decisions is 
a key part of the No Child Left Behind Act of 
2001. The evidence-based education that such 
initiatives promote involves the “integration 
of professional wisdom with the best available 
empirical evidence in making decisions about 
how to deliver instruction.” A wealth of data 
at the school, district, state, and federal levels 
should in principle provide an empirical basis 
for developing educational policies, practices, 
and research proposals and designs. 

The states in the Midwest Region are develop- 
ing innovative practices for identifying and 



addressing information priorities within their 
states and for meeting federal requirements. 
These exemplary practices involve establish- 
ing longitudinal student-level and teacher- 
level data collections and linking data across 
the educational information system. Other 
practices include incorporating key data ele- 
ments that can leverage other data resources to 
identify problems that could constrain student 
achievement and using accountability systems 
to target educational resources more efficiently 
and effectively. 

Midwest states also face challenges in meeting 
these goals. Data collection staff and resources 
for training at the local level are limited, and 
many states do not have enough staff with the 
skills and experience necessary to analyze the 
data. Keeping the duplication of data collec- 
tion to a minimum is also a constant challenge. 
Finally, federal and state regulations often con- 
strain states’ ability to collect key data elements. 

Given these challenges and constraints, 
responding to states’ information needs and 
aspirations may best be achieved through a 
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two-pronged approach. First is to establish 
regional benchmarks and provide guidelines 
for states wishing to use local data to develop 
indicators for purposes of comparison. Sec- 
ond is to respond to specific state requests 
for analytic resources and develop associated 
training materials. Both tasks have the explicit 



goals of providing immediate utility and 
building capacity for the future. Each may use- 
fully be addressed by the regional educational 
laboratories — singly, in combination, and with 
external partners. 

June 2007 
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States in the 
Midwest Region 
are developing 
innovative 
approaches to 
collecting and 
providing access to 
high-quality data 
in order to improve 
educational 
decisionmaking. 
Additional 
capacity-building 
and increased 
technical 
assistance at the 
state and local 
levels would 
enhance this work. 



Educational improvement through data-based 
decisionmaking using high-quality data is a 
longstanding goal of policymakers and practi- 
tioners, and ensuring the quality of the evidence 
available to inform such decisions is a key part 
of the No Child Left Behind Act of 2001 (NCLB). 



The evidence-based education that such initiatives 
promote involves the “integration of professional 
wisdom with the best available empirical evidence 
in making decisions about how to deliver instruc- 
tion.” A wealth of data at the school, district, state, 
and federal levels should in principle provide an 
empirical basis for developing educational policies, 
practices, and research proposals and designs. But 
the objectives of data-based decisionmaking in 
education have not been fully realized. 

Major factors contributing to this situation include 
data quality problems, outdated or incompatible 
systems and processes, organizational cultures 
that do not support data use for educational 
improvement, insufficient capacity to use multiple 
datasets efficiently, and a variety of organizational, 
logistical, and regulatory restrictions on making 
data — particularly individual-level student and 
teacher data — accessible to multiple audiences 
(Bernhardt, 2004; Massell, 2001; Streifer, 2004; 

U.S. General Accounting Office, 2005). There is 
thus a need to: 

• Assist state education agencies in enhancing 
the quality of state data. 

• Build capacity to leverage data to inform deci- 
sions and enhance strategic planning. 

• Provide technical and analytic assistance to 
states. 

This report describes the results of the first year 
of the REL Midwest Task 1.2 fast response project. 
Using Multiple Levels of Data to Address Educa- 
tional Issues in the Region, which seeks to address 
these needs (see box 1). Information obtained from 
the states in this study helps to define exemplary 
practices, common data problems, and analytic 
opportunities in the Midwest Region. 



EXEMPLARY PRACTICES 

The states in the Midwest Region 
clearly embrace — and several have 




2 



GETTING THE EVIDENCE FOR EVIDENCE-BASED INITIATIVES 



BOX 1 

Objectives of the study 

The objectives for this task were to: 

• Conduct a needs assessment to 
document current and expected 
priority information needs 

of each state in the Midwest 
Region, including information 
that would enable states to move 
beyond compliance with report- 
ing requirements toward more 
proactive strategic planning. 

• Develop and complete data 
inventories for each state in 



the Midwest Region to docu- 
ment the structure of the state’s 
educational data system and 
identify issues related to 
data quality, collection, and 
reporting. 

The approach was twofold (see ap- 
pendix A). First, key state education 
agency officials were interviewed 
about their state’s data system, 
including pressing requirements and 
current challenges. Second, state 
data inventories were completed to 
provide an overview of each state’s 
educational data system. To reduce 
the burden on respondents, these 



inventories were completed using in- 
formation publicly available in print 
and online formats. In addition, 
federal agency staff were contacted 
to provide additional perspectives 
for contextualizing issues identi- 
fied in the state education agency 
interviews, especially for federal- 
based initiatives, such as statewide 
longitudinal data systems (SLDS), 
the Education Data Exchange 
Network (EDEN), and data use and 
accessibility issues associated with 
state regulations and the Family 
Educational Rights and Privacy Act 
(FERPA). 



developed — innovative practices for identifying 
and addressing information priorities within 
their states and for meeting federal require- 
ments. Frequently these meet or exceed the 
additional steps that the Data Quality Campaign 
describes as fundamental to developing robust 
longitudinal data systems.^ Often they go beyond 
traditional practice, developing, adapting, or 
adopting innovative approaches to collecting, 
ensuring the quality of, and providing efficient 
and timely access to data for a range of planning 
and decisionmaking purposes. Many are what 
the Federal Emergency Management Agency 
characterizes as exemplary — ideas, projects, 
programs, techniques, or methods that have 
“worked in one place and may be worthy of 
adopting elsewhere.”^ 

Typically these exemplary practices involve estab- 
lishing longitudinal student-level and teacher-level 
data collections, linking data across the educa- 
tional information system, including key data 
elements that can leverage other data resources 
to assist in the early identification and treatment 
of problems that have the potential to constrain 
student achievement, and using accountability 
systems to target educational resources more ef- 
ficiently and effectively. 



Establishing individual-level longitudinal data systems 

All states in the Midwest Region have — or are 
in the process of establishing — individual-level 
student or teacher longitudinal data systems. 

In Michigan the Center for Educational Perfor- 
mance and Information, established in 2000, 
collects individual-level student data through 
the Single Record Student Database, one of five 
elements of the Michigan Education Information 
System data warehouse system. 

Iowa’s Project EASIER (Electronic Access System 
for Iowa Education Records) allows the electronic 
transfer of individual student data from school 
districts to the Department of Education to com- 
pile state and federal reports — and the exchange 
of student records between school districts when 
students transfer between schools in the state. 

The Department is working with postsecondary 
institutions to accomplish the electronic trans- 
fer of high school transcripts to postsecondary 
institutions. 

Indiana’s Student Test Number system, opera- 
tional since the 2002/03 school year, has been used 
to collect a variety of student-level longitudinal 
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data, including demographic information, 
enrollment, special program participation, drop- 
out, graduation, and other achievement data. 

Since the 2004/05 school year Wisconsin has 
assigned unique statewide student identifiers, 
providing the capacity to link student-level records 
across all the state’s student-level databases. 

Minnesota also assigns unique student identi- 
fiers, and in 2005 received — along with the states 
of Michigan and Wisconsin and the Wisconsin 
Center for Education Research — a federal State- 
wide Longitudinal Data Systems (SLDS) grant to 
support developing a comprehensive multistate 
longitudinal data system. 

In Ohio a third party vendor assigns unique 
statewide student identifiers to public school stu- 
dents; these identifiers can be linked to individual 
students only at designated data acquisition points, 
protecting student privacy while enabling longitudi- 
nal tracking of student performance across the state 
using data submitted through Ohio’s Education 
Management Information System. In 2006 the Ohio 
Department of Education in collaboration with 
local education agencies, regional information tech- 
nology centers, and other entities received funding 
from the SLDS program to support the Data-Driven 
Decisions for Academic Achievement (D3A2) proj- 
ect, increasing to four the number of states in the 
Midwest Region that have received SLDS funding 
(of 14 awards under that program to date). 

In 2005/06 Illinois began to implement the state 
Student Information System, which assigns unique 
student identification numbers to each public 
school student in the state. This system also collects 
and stores demographic, graduation, dropout, and 
other information. Although the Illinois Student 
Information System does not store any informa- 
tion on teachers, its Teacher Service Record collects 
information on current positions and assignments 
for all teachers currently employed in the state. 



longitudinally. Typically the establishment of 
student-level systems is given priority over teacher/ 
stair systems, although many states have made 
significant strides in this realm as well. As noted 
above, several have obtained outside funding to 
support the development of student-level longitu- 
dinal data systems; others have worked to develop 
student-level systems by reallocating resources. 
Several states have also taken steps to begin what 
are generally envisaged as multi-year efforts 
to establish data warehousing systems (Iowa, 
Minnesota, Michigan, Ohio). Others are eager to 
move in this direction (Indiana, Illinois), building 
steps to achieve this goal into their strategic plans 
and making requests for funding to state legisla- 
tive bodies. With the establishment of such data 
enclaves, states in the Midwest Region will move 
closer to attaining a series of shared objectives: to 
link data at the state level and to make it easier for 
local entities to add value with their own local data 
to enhance decisions about educational policies 
and practices not only at the state level but also at 
the district, school, and classroom levels. 



Linking data across the educational information system 



Implicit in the attainment of the state’s most 
proactive objectives for their data is the ability to 
link data from multiple sources. An obvious (but 
for many states currently 
unattainable) goal is the 
ability to place students 
in their classrooms with 
their teachers — that is, to 
link student and teacher 
data. Several states hold 
multiple years of student- 
level demographic, 

enrollment, and achievement data — all of which 
can be linked — but have no system to easily link 
teacher with student data. Obstacles to establish- 
ing these links include addressing teacher unions 
concerns with the confidentiality of individual 
teacher data. 



An obvious (but for 
many states currently 
unattainable) goal is the 
ability to place students 
in their classrooms 
with their teachers 



Clearly, all states in the region recognize the 
importance of access to individual-level data 



Another common objective is to extend the 
utility of preK-12 data by developing links with 
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postsecondary education data systems. While 
several states have plans in place or are develop- 
ing strategies to address this priority, constraints 
include: 



• The absence of a common identifier across 

student-level K-12 and postsecondary record- 
keeping systems. 



• The absence of a suitable data warehousing infra- 
structure. Several states are conducting reviews to 
identify opportunities to develop systems to gather 
such data; the results of these reviews may well 
prove valuable to other states in the region and the 
country overall. 



Frequently states seek 
education data not just 
for themselves but for 



Another common objective is to enrich existing 
data collections with additional data elements, 
such as course-taking and course-completion data 
for students, and for teachers, details of their pre- 
and post-certification training and professional 
development activities. The former can prove 
essential in understanding how particular forms 
of instruction and course- sequences affect edu- 
cational outcomes; the latter can 
be important in identifying and 
targeting effective professional 
development practices. 



individual teachers and 
their principals, enabling 
practitioners to see 
what best practices are 
characteristic of their 
schools, the factors 
promoting their use, and 
their ultimate outcomes 



Developing diagnostics 



The goal of developing mecha- 
nisms and procedures to link 
student with teacher or student 
preK-12 with student higher 
education data is to identify, di- 
agnose, and intervene to remedi- 
ate situations that, unchecked, 
are likely to lead to undesirable student learning 
outcomes. Frequently states seek this information 
not just for themselves but for individual teach- 
ers and their principals, enabling practitioners to 
see what best practices are characteristic of their 
schools, the factors promoting their use, and their 
ultimate outcomes. States seek to forge preK-12 
links with postsecondary data in order to high- 
light high school experiences and clarify which 



high schools are better at preparing their students 
for college — and why. They expect to link student 
with teacher data to identify teacher professional 
development practices that do (and do not) lead 
to improved student achievement and to out- 
standing instructors and instructional practices. 
Another goal is to link teacher preparation with 
teacher mobility and attrition data in an effort to 
understand why instructors from some of the best 
teacher preparation programs and courses decide 
to leave the teaching profession at various stages in 
their careers. States also seek to enhance student- 
level record systems with course-taking and 
course-completion data to assist district officials 
and teachers in understanding the relationship 
between classes, courses, and performance. 

Several states have identified key data elements 
that will help with early identification of problems 
(for example, course-taking and course- comple- 
tion, family involvement, and school climate). In 
some cases this data already exists within the state 
system and simply needs to be reported; in others, 
additional data collection may be required. 



Using accountability data 

Several states seek to go beyond the accountability 
requirements of the No Child Left Behind Act and 
other state and federal mandates, using account- 
ability data to monitor progress and target re- 
sources more efficiently and effectively. Examples 
include initiatives to: 

• Identify schools that are making significant prog- 
ress (such as moving students from the bottom to 
the next quartile), even if they are not necessar- 
ily moving all students to proficiency or making 
adequate yearly progress. 

• Identify how funds are being allocated (such as 
supporting teacher professional development). 

• Establish the differences such resources are 
making on intermediate and final educational 
outcomes (such as the impacts of professional 
development activities on the career development 
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of individual teachers, on particular categories 
of teachers, and on the educational attainment of 
their students). 

• Target resources more efficiently (say, on individu- 
alized education programs and limited English 
proficient activities and programs). 

• Use achievement data in a more timely fashion 
(supporting continuous assessment within the 
classroom). 



CHALLENGES AND CONSTRAINTS 

The foregoing exemplary practices notwithstand- 
ing, efforts to address information needs across 
the Midwest Region are frequently hampered by 
a combination of factors. Common challenges 
include analytic capacity, staff, and other resource 
constraints; data burden; and concerns with the 
implications of the Family Educational Rights 
and Privacy Act and related state and federal 
regulations. 



Analytic capacity, staff, and resource constraints 

Several states commented on the difficulty of 
conducting noncompulsory analyses of existing 
data given current staff resources. Some states 
have had to cut staff— accomplishing more tasks 
with fewer resources. Others have tackled the 
challenges of moving to individual-level data 
collection using resources originally designed for 
aggregate data collection. A common challenge is 
to recruit and retain skilled analysts given salary 
differentials in public education and other sectors. 
Turnover at the local level also contributes to 
state-level resource constraints. When training 
in the use of new data collection and reporting 
systems is accomplished by reallocating state 
education agency officials’ time to serve as local 
trainers, high turnover at the school and district 
level has implications for state education agency 
staff. Such problems could be resolved by hiring 
additional skilled analysts or providing addi- 
tional professional development for existing staff. 



Another option is to provide external analytic 
capacity. 



Data burden 



States in the region seek to provide both state and 
local actors with specific information to guide 
policy development and practice. Several initia- 
tives are already in place and others are in the 
planning stage. Even so, many of these aspirations 
have yet to be realized. Factors currently affecting 
the ability of states to collect the data necessary to 
meet these objectives include: 



• An inability to provide resources to local officials 
(at the school or district level), on whom consid- 
erable extra burden is placed with a move from 
aggregate to individual reporting of data elements. 

• The absence of a legal authority to collect addi- 
tional data. 



Avoiding duplication in data collection. One way 
to reduce the burden is to restrict (if not elimi- 
nate) duplicative data collections. A key strategy 
for achieving this objective in several states in 
the region is establishing data warehouses and 
integrating records in databases that can be que- 
ried to extract information for multiple purposes 
Such systems can be stimulated by a state educa- 
tion agency data culture (Michigan’s educational 
data mission is “to collect once, store once, and 
use many times”). Or 
they can be pursued in 
response to legislative 
mandate (Michigan’s 
Public Act 180 of 2003 
authorizes the Center 
for Educational Perfor- 
mance and Informa- 
tion to coordinate data 
collection in an effort to 
reduce districts’ report- 
ing burden and prevent 

duplicate data collections). The expected result: 
more efficient and effective data collection, stor- 
age, querying, and reporting capacities. An added 



A key strategy for 
reducing data burden 
is establishing data 
warehouses and 
integrating records 
in databases that can 
be queried to extract 
information for 
multiple purposes 
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benefit of such a strategy is the reduced likelihood 
of introducing inaccurate data into the system 
(assuming appropriate quality control and reason- 
ability checks are in place to ensure the validity of 
data as it is added to the system). 

Data burden is also an issue when requests to 
report the same data in multiple forms (as in 
compliance with multiple federal collections and 
in response to state and district officials’ and other 
stakeholders’ information requests). Initiatives 
already under way at the federal level to address 
some of these issues (such as streamlined collec- 
tion of CCD and compliance data through EDEN) 
may free some scarce staff resources for realloca- 
tion to other state data needs and aspirations. 



A culture is developing 
within states and across 
the region in support of 
sharing data to inform 
decisionmaking at 
multiple levels in the 
educational system while 
minimizing the burden 
of multiple collections 



Establishing legislative authority to collect data. 
Another significant challenge to leveraging the full 
benefits of data already collected is the inability to 
enrich that information with additional data that 
could be used to develop causal inferences regard- 
ing the factors contributing to various student 
learning outcomes. Key here are statutory require- 
ments that can preclude the collection of data at 
the state level without specific legislative author- 
ity. An example is Indiana, a local control state. 

The state constitution provides that any authority 
not specifically given to the central government 
reverts to the local level— in the case of education, 
the school boards are the legal entities controlling 
the educational system. As a result, all educational 
data reporting is completely voluntary unless 
specifically required by a state board rule or state 
law. Similarly in Michigan data collection activi- 
ties must be state or federally mandated; the state 
has no authority to collect data not 
required for compliance with such 
mandates. 



Implications of the Family 
Educational Rights and Privacy 
Act and state regulations 

While acknowledging the impor- 
tant protections under the Family 
Educational Rights and Privacy 



Act and other federal (such as the Protection of 
Pupil Rights Amendment, PPRA"*) and state regu- 
lations, several states underscored the constraints 
such regulations place on their ability to extract 
the full value of the data they already collect for 
other purposes. Concerns here include the ability 
to return to districts and schools data collected 
and processed by the state, and the ability of K-12 
and higher education institutions to share data 
while still protecting the rights of both students 
and their families. A culture is developing within 
states and across the region in support of sharing 
data to inform decisionmaking at multiple levels 
in the educational system while minimizing the 
burden of multiple collections. States support 
allocating resources to these efforts — but remain 
concerned that these goals are unachievable given 
FERPA and other regulations. 



ANALYTIC POSSIBILITIES 

Given the challenges and constraints, responding 
to states’ expressed information needs and aspira- 
tions may best be achieved through a two-pronged 
approach. First is to establish regional benchmarks 
and provide guidelines for states wishing to use 
local data to develop comparable indicators for 
purposes of comparison. Second is to respond to 
specific state requests for analytic resources and de- 
velop associated training materials. Both tasks have 
the explicit goals of providing immediate utility and 
building capacity for the future. Each may usefully 
be addressed by the regional educational labora- 
tories — singly, in combination, and with external 
partners. Here are suggestions for steps that REE 
Midwest might usefully take to provide additional 
analytic support to the states in this region. 



Providing regional benchmarks 

In the absence of long-standing longitudinal 
student or teacher data at the state level, it can 
be challenging to undertake the trend analyses 
required to establish appropriate benchmarks for 
establishing performance objectives, assessing and 
understanding factors influencing outcomes, and 
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selecting or developing interventions to enhance 
performance. Initiatives under way in several 
states in the Midwest Region to develop statewide 
longitudinal data systems will provide the data 
necessary to develop key state-level indicators 
in the future. While such systems continue to 
mature, several priority information needs and 
aspirations can be addressed through analyses of 
major secondary sources — alone or in combina- 
tion with existing state data. Examples include: 

• Providing benchmarks for monitoring growth 
in student achievement (including comparisons 
across similar types of students, and for students 
enrolled in similar types of schools) both nation- 
ally and in the Midwest Region. 

• Providing benchmarks for monitoring states’ 
performance relative to other states in the Midwest 
Region, nationally, and internationally. 

• Providing benchmarks to assist states in assessing 
the performance of specific subgroups or subpopu- 
lations of students. 

• Identifying teacher professional development and 
other factors associated with teachers’ job satisfac- 
tion and high growth in student achievement. 



Responding to requests for analytic resources: 

Addressing immediate information needs 
while building capacity for the future 

This study suggests numerous areas where states 
would benefit from access to analytic resources 
that would assist them in resolving information 
needs to move beyond compliance with federal 
and other reporting requirements toward more 
proactive strategic planning for educational 
process, system, and outcome improvements. 

One example is creating an “at-risk” profile for 
high school students. A second is using existing 
data dictionaries to create metadata structures. A 
third is identifying key outcomes associated with 
various teacher professional development activities 
and opportunities and factors affecting teacher 
mobility and retention rates (say, by district). 



Another source of support would be to develop 
associated training materials that would provide 
professional development capacity building re- 
sources to interested states. Such training materi- 
als might specify the data elements and required 
data quality, methodologies, analytic approaches, 
and procedures for resolving specific informa- 
tion requirements using available state data. They 
might also specify the procedures for resolving 
similar information requirements using data 
states anticipate having access to in the future, as 
longitudinal data systems mature and technical 
and regulatory issues affecting the ability to link 
data from multiple sources for particular purposes 
are resolved. 



Facilitating a comprehensive, systematic approach to 
realize the objectives of data-driven decisionmaking 

The No Child Left Behind Act is a push toward 
creating the individual-level accountability data 
systems that can be used to inform decisions on 
educational policies and practices at the federal, 
state, district, school, and classroom levels. The 
states seek to do much more than meet such com- 
pliance requirements. They recognize the exciting 
and important opportunities individual-level 
longitudinal data systems create to be proactive, 
to identify at-risk populations, to target interven- 
tions earlier, and to monitor their impacts, quickly 
adjusting policies and programs as required. 

Many states in the region, having already gone far 
beyond recognizing these opportunities, are devel- 
oping plans and committing resources to achieve 
them. With different opportunity structures and 
different experiences addressing these issues, they 
have the potential to provide considerable practical 
experience to each other. State education agency 
officials in the region have much in common in 
their objectives for their educational data systems. 
This analysis suggests important opportunities 
exist to capitalize on the states’ commitments to 
data-driven decisionmaking and enhance their 
opportunities to realize its potential by think- 
ing about these issues more comprehensively and 
systematically. 
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NOTES 

1 . 



dataqualitycampaign.org/survey_results/ 
fundamentals. cfm. 



See “Evidence-Based Education (EBE),” June 9, 
2003 presentation by Grover J. (Russ) White- 
hurst, then Assistant Secretary, Educational 
Research and Improvement, United States 
Department of Education, available online at 
http://www.ed.gov/offices/OERI/presentations/ 
evidencebase.ppt (link verified March 9, 2007). 

See “Eundamentals in Designing State Longi- 
tudinal Data Systems,” online at http://www. 



3. See Partnerships in Preparedness: A Compen- 
dium of Exemplary Practices in Emergency 
Management, Volume II - May 1997, available 
online at http://www.fema.gov/emergency/ 
managers/partnr02.shtm. 

4. See http://www.ed.gov/policy/gen/guid/fpco/ 
ppra/index.html. 
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APPENDIX A 
METHODS 

Instrumentation developed for the needs assess- 
ment portion of this study included an in-depth 
interview protocol and informed consent forms. 
The protocol was designed for administration in 
personal (face-to-face or telephone) interviews of 
approximately 90-minutes duration with senior 
state education agency officials. Respondents are 
asked to: (1) describe data collection processes and 
reporting practices within the region, (2) identify 
state education agencies’ pressing information 
needs, and (3) identify data and other informa- 
tion resources which would be most beneficial to 
the state in utilizing data to enhance proactive 
strategic planning. Specific topics addressed in the 
interview include: respondents’ roles and respon- 
sibilities in defining state data needs; questions 
states would like their educational data to address; 
states’ most pressing information needs; any chal- 
lenges states may currently be facing or anticipat- 
ing in addressing these information needs; the 
extent to which states may be experiencing com- 
mon challenges faced by large data collection and 
information processing systems; and resources 
available to address state information needs. 
Instrumentation was developed and interviews 
and analyses were conducted by senior research- 
ers at Michigan State University and NORC at the 
University of Chicago. 

The original proposal called for one senior state 
education agency official from each of the seven 
states to be interviewed. REL Midwest Executive 
Director Steven Cantrell introduced this study to 
chief state school officers, five of whom identified 
individuals to serve as respondents for this study. 
Four states (Illinois, Indiana, Iowa, and Wiscon- 
sin) identified one respondent; a fifth (Ohio) iden- 
tified two respondents. Two states (Michigan and 
Minnesota) did not specify respondents; senior 
laboratory staff offered guidance in identifying 
appropriate contacts in each of these state agen- 
cies. A total of eight interviews were conducted. 
While facilitating expedited data collection in 
keeping with the timeframe of Task 1.2 studies 



(surveying 10 or more individuals with identical 
questions would have required approval from the 
Office of Management and Budget in keeping with 
the provisions of the Paperwork Reduction Act), 
a resulting limitation of the study is that in most 
states interviews were conducted with a single 
person. Each person was invited to participate on 
the basis of acknowledged expertise in the area 
(respondents were typically identified by their 
chief state school officers as the most knowledge- 
able state education agency officials on the topics 
addressed in the in-depth interviews); the perspec- 
tives provided were those of that individual. All 
instrumentation and procedures pertaining to 
these interviews were reviewed and approved by 
NORC’s Institutional Review Board (IRB Protocol 
Number 061002). 

The data inventories were designed to provide 
information on the data collection activities in 
each of the states, including what types of data 
are collected; how often the data are collected; the 
agencies and stalf involved in collecting, main- 
taining, and providing access to the data; where 
the data are stored and made accessible; when 
data elements were first collected; and whether el- 
ements were subsequently re-coded or re-defined 
in ways that might influence longitudinal analy- 
ses. The data inventories were completed using a 
two-step process. Results from the Survey of State 
Data Collection Issues Related to Longitudinal 
Analysis, conducted by the Data Quality Cam- 
paign and the National Center for Educational 
Accountability^ served as the initial resource for 
information on the types of data collected by 
and available for state use. This survey of state 
education officials has been conducted since 2003 
and includes questions on ten components the 
Data Quality Campaign characterizes as essen- 
tial to building a state longitudinal student data 
system (see http://www.dataqualitycampaign.org/ 
essential_elements.cfm): 

A unique statewide student identifier. 

Student-level enrollment, demographic and pro- 
gram participation. 
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TABLE A1 

Student and teacher data elements about which information is recorded in the state data inventories 





IL 


IN 


lA 


Ml 


MN 


OH 


wi 1 


Student data elements 


Demographic data 


Gender 


/ 


/ 


/ 


/ 


/ 


/ 


/ 


Date of birth 


/ 


/ 


/ 


/ 


/ 


/ 


/ 


Ethnicity 


/ 


/ 


/ 


/ 


/ 


/ 


/ 


Race 


/ 


/ 


/ 


/ 


/ 


/ 


/ 


Low-income status 


/ 


/ 


/ 


/ 


/ 


/ 


/ 


Special population data 


English proficiency status 


/ 


/ 


/ 


/ 


/ 


/ 


/ 


Disability status 


/ 


/ 


/ 


/ 


— 


/ 


/ 


Economic disadvantage 


/ 


/ 


/ 


/ 


/ 


/ 


/ 


Migrant status 


/ 


/ 


/ 


/ 


— 


/ 


/ 


Enrollment data 


School 


/ 


/ 


/ 


/ 


/ 


/ 


/ 


Mobility 


/ 


/ 


/ 


/ 


/ 


/ 


/ 


Attendance 


— 


/ 


/ 


/ 


/ 


/ 


/ 


Curriculum data 


Courses completed 


— 


— 


/ 


— 


— 


/ 


/ 


Course numbering system 


— 


/ 


/ 


— 


— 


/ 


/ 


MS courses for HS credit 


— 


— 


— 


— 


— 


/ 


— 


Dual enrollment courses 


— 


— 


/ 


— 


— 


/ 


— 


Achievement data 


State test scores 


/ 


/ 


/ 


/ 


/ 


/ 


/ 


Data permanently stored 


/ 


/ 


/ 


/ 


/ 


/ 


/ 


Untested student records 


/ 


/ 


/ 


/ 


/ 


/ 


/ 


Special program data 


Early childhood 


/ 


/ 


/ 


/ 


— 


/ 


/ 


Individualized education 


/ 


/ 


/ 


/ 


— 


/ 


/ 


Special assistance 


/ 


/ 


/ 


/ 


— 


/ 


/ 


Honors courses 


— 


/ 


— 


/ 


— 


/ 


/ 


College test score data 


SAT scores 


— 


— 


— 


/ 


— 


— 


— 


ACT scores 


— 


— 


/ 


/ 


— 


— 


/ 


AP exam scores 


— 


— 


— 


— 


— 


— 


/ 


Attainment data 


HS graduate 


/ 


/ 


/ 


/ 


/ 


/ 


/ 


Type of diploma 


— 


/ 


/ 


/ 


/ 


/ 


/ 


School dropout 


/ 


/ 


/ 


/ 


/ 


/ 


y 


Dropout follow-up 


— 


/ 


/ 


/ 


— 


— 


— 


Dropout vs. transfer 


/ 


/ 


— 


/ 


— 


/ 


— 
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TABLE A1 (CONTINUED) 

Student and teacher data elements about which information is recorded in the state data inventories 





IL 


iN 


iA 


Mi 


MN 


OH 


Wi 1 


Follow-up standards 


— 


— 


/ 


— 


/ 


— 


/ 


Teacher data elements 


Demographic data 


/ 


/ 


/ 


/ 


/ 


/ 


/ 


Qualifications data 


/ 


/ 


/ 


/ 


/ 


/ 


/ 


Professional development 


— 


— 


— 


/ 


— 


— 


— 


Mobility data 


— 


— 


— 


/ 


— 


/ 


— 


Attrition data 


— 


— 


— 


/ 


— 


/ 


— 



Capability to match student test scores over time. 

Information on untested students. 

A teacher identifier system to match teachers to 
students. 

Student-level transcription information. 
Student-level college readiness test scores. 
Student-level graduation and dropout data. 



mobility, student special program participation, 
teacher demographics, teacher certification and 
qualifications, teacher professional development, 
teacher mobility, and teacher attrition. 

Table A1 illustrates the type of information re- 
corded in the state data inventories with respect to 
student and teacher data elements collected by the 
seven states in the Midwest Region. Specifically, 
each cell containing a check mark (/) indicates 
a data element about which information was ob- 
tained and is recorded in each state’s inventory. 



Capability to match student records between P-12 
and higher education systems. 

A state data audit system. 

Responses to the most recent survey^ (circa Sep- 
tember 2006) were cross-checked with information 
available on state education agency web sites and 
obtained from state education agency officials. 

Additional items with no counterpart on the NCEA 
survey (for example, when data elements were first 
collected, agencies and staff involved in their col- 
lection, changes in data definitions and/or response 

categories over time) included in the state data Notes 

inventories developed for this study were similarly 
populated using publicly available documentation 
(including instrumentation, coding protocols, and 
management information systems state educa- 
tion agencies make available online). The resulting 
inventories provide complementary information 
about student special population status, student 



The inventories also provide additional informa- 
tion on data quality control procedures, links 
to related online resources (such as education 
management information system manuals, data 
collection instruments, and data dictionaries), and 
contact information for key state education agency 
officials. Key information from the state data 
inventories is summarized in the individual state 
reports appearing in appendices B-H. States in 
the Midwest Region can access their full inventory 
through a password-protected web site; instruc- 
tions for accessing this site are in Appendix I. 



1. The National Center for Educational Account- 
ability (NCEA, online at http://www.just4kids. 
org/en/) is a non-profit, non-partisan orga- 
nization and national sponsor of Just for the 
Kids “which provides school, district, and 
other education leaders with essential system 





12 



GETTING THE EVIDENCE FOR EVIDENCE-BASED INITIATIVES 



performance and best practice information 
they need to make sound decisions” (http:// 
www.just4kids.org/en/jftk/). The Data Qual- 
ity Campaign (DQC, online at http://www. 
dataqualitycampaign.org/), managed by the 
NCEA, “was created in 2005, with support 
from The Bill & Melinda Gates Foundation, 
as a way for many organizations who were 
working on separate but similar campaigns 
regarding educational data systems to come 
together to ensure coordinated and undupli- 
cated efforts towards reaching their common 
goals.” DQC’s mission is “to provide support 



for and advocacy on behalf of organizations 
that create, collect, and use education data in 
an effort to improve student achievement,” 
(see http://www.dataqualitycampaign.org/ 
about_us/). 

2. The 2006 DQC-NCEA survey instru- 
ment is available online at http:// 
www.dataqualitycampaign.org/files/ 
Survey-Sample_2006_111306.pdf; re- 
sults are available in a variety of formats 
on the DQC web site via http://www. 
dataqualitycampaign.org/survey_results/. 
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APPENDIX B 
ILLINOIS 



Information needs assessment 

Data collection and quality assurance procedures 

The Data Analysis & Progress Reporting Division of 
the Illinois State Board of Education (ISBE) has pri- 
mary responsibility for collecting education data in 
Illinois. This division also is responsible for analysis, 
report writing, and submitting performance reports 
to the U.S. Department of Education. The division 
administrator is aided by approximately eleven staff 
members and is supported by the Data Systems 
Division, which maintains the data collected. 

The division began implementing the Student 
Information System (SIS) in 2005/06. The SIS is 
meant to consolidate and build on a multitude of 
other data collection efforts within the ISBE. All 
but three school districts (including Chicago) are 
participating in the SIS, which should be fully 
implemented by 2008/09. 

The data are audited both by the ISBE Internal 
Audit Division and by the federal government. The 
Illinois’ Auditor General and Comptroller’s Office 
also take yearly random samples of ISBE data col- 
lection efforts from the district level up through 
online processing to see how the data are edited, 
checked for errors, processed, analyzed, and dis- 
seminated. The ISBE does not currently designate 
staff with specific responsibility for monitoring 
data quality, nor is state staff given specific train- 
ing in data collection practices and reporting. 

More information about ISBE data collection can 
be obtained on the SIS web site at http://www.isbe. 
state.il.us/sis/default.htm. 

Defining state data needs 

How Illinois defines data needs 

Data needs in Illinois are largely defined either 
by state or federal mandates or requirements. The 



Governor, Illinois General Assembly, and nine 
member State Board of Education set state require- 
ments for data collection. The Data Analysis & 
Progress Reporting Division works closely with 
the various grant program managers to ensure 
that required data are collected and with the Gov- 
ernmental Relations Department to ensure that 
state and federal requirements are met. Prelimi- 
nary discussions also are under way with the Il- 
linois Board of Higher Education and Department 
of Human Services to collect and share data. 

Information and data priorities 

A pressing need in Illinois is to compile all the 
available information into a data warehouse and 
streamline the process of warehousing data. The 
latter would simplify data collection efforts and 
increase the accuracy of data collected, while the 
former would make the information more use- 
ful since it could be queried, or mined at least, 
to address various types of questions. The most 
important of these questions include whether or 
not students are actually learning, at what rate, 
and with which teachers. 

Data currently available can be used to address 
questions concerning populations of students 
such as schools or districts. Student performance 
data can be put in the context of the teaching 
resources and finances available at the school or 
district level. Recent studies have looked at the 
relationship between school poverty levels and 
performance. 

The remainder of this section reviews the kinds of 
questions the state of Illinois would most like its 
educational data to address in the short-term and 
looking further to the future, and highlights the 
information needs which senior state education 
agency officials currently consider to be particular 
priorities. Existing challenges which may affect 
the state’s ability to achieve these objectives mov- 
ing forward are discussed below. 

Establishing benchmarks. Comparing Illinois with 
the rest of the nation is a high priority, especially 
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given current plans for ranking states at the federal 
level. Other priorities that relate to No Child Left 
Behind include the performance of limited English 
proficient (LEP) students and the performance of 
individualized education program (lEP) students. 
Although compiling data on these subgroups has 
created some unexpected tensions among students 
and parents, the ISBE needs this data to focus 
resources on the students that really need the help. 

Tracking progress. Measuring progress in student 
learning is a top priority in Illinois. However, 
the state has had difficulty obtaining assessment 
results in a timely fashion. A new vendor did not 
pass along final 2005/06 test results for grades 
3 through 8 in time to do School Report Cards, 
nominate blue ribbon schools to the federal gov- 
ernment, upload state data to EDEN, and report 
on Reading First results or create other reading 
improvement reports. The vendor was fined, and 
external firms were hired to monitor and assist the 
vendor to make sure that the testing system is in 
place and running smoothly. 

The ISBE would like more information about 
teacher training, which most districts do not 
collect or collect only in relation to district-wide 
professional development efforts. Teacher unions in 
Illinois also resist efforts to link teachers to learn- 
ing outcomes. Better data on graduation rates also 
is a priority and is part of the SIS. Previous calcula- 
tions relied on cohort analysis of self-reported data 
from the local school districts. Being able to track 
students through college is under consideration, 
but FERPA requirements may limit the ability of 
the ISBE to share student information with the Il- 
linois Board of Higher Education (IBHE). 

Documenting outcomes and their causes. As noted 
above, tracking students through college and 
linking outcomes to teachers are areas of interest 
to the ISBE, but significant barriers prevent the 
collection of such data. Lower priority issues in the 
Division include data on dropout rates, which have 
remained stable, or on curriculum development, 
which largely is under local school control. An out- 
come currently being studied is district and school 



implementation of the Illinois learning standards 
for reading and mathematics. High schools feel less 
pressure to implement these standards as long as 
they are performing well on state tests. 

Enhancing capacity to use existing data resources. 
The ISBE currently collects data using the IWAS 
system. Hardware limitations on the analysis of 
existing data include lack of a mainframe com- 
puter for the division and outdated PCs with 
insufficient processing power. More staff in data 
systems and analysis would be helpful, although 
the state caps employment and hiring priorities 
often are set by the assistant superintendents. Of- 
fering competitive salaries also is an issue even if a 
division is authorized to hire. 

Utilization of data would be greatly improved if all 
information collected throughout the ISBE were 
warehoused in a single relational database. The 
school Report Card, for example, combines eleven 
different data files controlled by four different di- 
visions. As mentioned above, the ISBE is exploring 
the cost of constructing a data warehouse but has 
not allocated funds to build it.' 

Understanding state data challenges 

Educational data are being used in new ways in Il- 
linois since the passage of NCLB. District, schools, 
and students face real consequences if their assess- 
ments fall short of federal standards. At the state 
level this means that decisions ranging from the 
provision of supplemental educational services to 
restructuring entire schools are tied directly to 
educational data, making the collection and accu- 
racy of this information a top priority for everyone 
involved. As a result, the Data Analysis 8c Progress 
Reporting Division has much more responsibility 
and more demands in terms of data collection and 
reporting. The audiences have become more varied 
too, as has the type of data reported (e.g. disaggre- 
gated by student subgroups such as LEP and lEP). 

Overall, this new emphasis on data has posi- 
tively affected the educational process by allow- 
ing districts to see which groups of students are 
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underperforming so that they can focus limited 
resources to try to bring about improvement. There 
have been some unintended negative consequences, 
however. Some districts blame lEP students for 
schools not making adequate yearly progress, while 
teachers are made to feel like failures when their 
school is designated as “failing.” Illinois also is hav- 
ing trouble meeting all of the new federal reporting 
requirements on time. Part of the problem is that 
large school districts like Chicago have understaffed 
data collection efforts and suffer from extremely 
high student turnover, meaning that required data 
are six to eight months late. District staff turnover 
is a problem too given the very intensive training 
recently provided to those who are responsible for 
submitting data to the state. Whatever the causes, 
Illinois may end up withholding funds from dis- 
tricts if the federal government starts fining states 
for not submitting their data on time. 

Without the SIS in place, other challenges cur- 
rently facing the ISBE include redundancy in data 
collection efforts and inconsistencies in the treat- 
ment of missing data. Enrollment or “housing” 
data, for example, is collected separately several 
times over the course of a school year, meaning 
that reports will use different data depending on 
when they are produced. Efforts to get the Illinois 
General Assembly to modify the data collection 
timeline were unsuccessful. The treatment of 
missing data is a problem since multiple contrac- 
tors may collect the same data, such as assess- 
ment, using different methods of coding. This is 
a problem because the state has to follow up with 



districts in order to get them to submit the data 
that is coded as missing. 

Implementation of the Student Information System 
along with a data warehouse would address many 
of Illinois’ data collection challenges. This would 
allow the state to make more decisions about 
schools and students based on solid information 
rather than on hearsay. For example, better data on 
the number of LEP students in the state would let 
legislators know how much of the budget needs to 
be earmarked for programs to help those kids. So 
the return on investing in educational data in Illi- 
nois would be more data-driven policy decisions. 



State data inventory 

Illinois' educational data system 

Responsibility for collecting data in the Illinois 
State Board of Education falls under the Data 
Analysis & Progress Reporting Division and the 
Student Assessment Division, both of which are 
under the Department for Teaching & Learning 
Services for All Children. 

Table B1 gives contact information for the heads of 
the two divisions primarily responsible for collect- 
ing and warehousing data. 

Key data elements 

According to the 2006 NCEA Survey of 
State P-12 Longitudinal Data Systems (see 



TABLE B1 

Key contacts for more information on Illinois' data processes and files 



1 These data collection activities . . . 


... are currently the responsibility of: 


Analyzing data for policy and planning, and coordinating 
annual reporting on progress related to Board goals and 
legislative requirements 


Connie Wise, Division Administrator 

217-782-3950 

cwise@isbe.net 


Developing and administering tests that measure the 
performance of students and schools against the Illinois 
Learning Standards 


Becky McCabe, Division Administrator 

217-782-4823 

rmccabe@isbe.net 


Developing a data warehouse 


Terry Chamberlain, Data Systems Administrator 

217-782-4313 

tchamber@isbe.net 
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http://www.dataqualitycampaign.org/survey_ 
results/state.cfm?st=Illinois), Illinois is one of six 
states nationwide to have only one to three of the 
elements deemed essential to building a robust 
longitudinal student data system. Starting in 2006, 
the state assigns a unique student identification 
number which is used to collect and store stu- 
dent demographic data and the school in which 
the student was enrolled. However, Illinois does 
not (1) currently match individual students’ test 
records from year to year, (2) match teacher data 
to student data, (3) collect and maintain student- 
level course completion data, (4) permanently 
store student-level college readiness test scores, (5) 
track graduates into postsecondary education, or 
(6) have an audit system that ensures the validity 
of reported data. In contrast to the NCEA Survey, 
online research determined that Illinois does seem 
to collect student-level graduation and dropout 
data, which would give the state a fourth essential 
element. 

The remainder of this section provides an over- 
view and highlights of student and teacher data 
currently collected and archived by the state of 
Illinois. For additional information, see the web 
site of the Student Information System that is cur- 
rently under construction as a data warehouse for 
the ISBE at http://www.isbe.net/sis/default.htm. 

Student data 

Demographics. The ISBE began collecting indi- 
vidual student demographic data in the 2005/06 
school year. The data are collected at least twice 
yearly and include information on student gender, 
date of birth, ethnicity, low-income status, English 
language proficiency, disability status, and mi- 
grant status. 

Enrollment. The ISBE began collecting individual 
student enrollment data in the 2005/06 school 
year. The data are collected at least twice yearly 
and include information on where a student is 
enrolled and the dates of enrollment. Student mo- 
bility is indicated by “entry” and “exit” enrollment 
data. These data are permanently stored. 



Course-taking. The ISBE does not collect any data 
on the courses that a student completes. 

Special program participation. The ISBE began 
collecting data on student participation in special 
programs in the 2005/06 school year. The data 
are collected at least twice yearly and include 
information on early childhood program par- 
ticipation, individualized education program 
information, and special assistance programs 
(e.g. 21st Century, Reading 1st, Title I). Illinois 
does not collect data on honors or AP program 
participation. 

Achievement. The ISBE has been collecting student 
achievement data since the 1999/2000 school year. 
Standardized state tests include the Illinois Stan- 
dards Achievement Test (ISAT) and the Prairie 
State Achievement Examination (PSAE). Collec- 
tion of data on untested students began in the 
2006/07 school year; these data are stored. Illinois 
does not collect and store student-level college- 
readiness test scores (e.g. ACT, SAT). 

Graduation and dropout data. The ISBE began 
collecting student-level graduation data in the 
2005/06 school year, though not by type of diploma 
received. The state also began collecting individual 
student dropout data during the same year, includ- 
ing the ability to categorize departing students as 
“dropouts,” “transfers,” or “missing.” Illinois does 
not collect information on where “dropouts” went. 

Teacher data 

Demographics. The ISBE has been collecting data 
on individual teacher demographics at least since 
the early 1960s, including school ID, gender, 
ethnicity, subject assignment(s), position title, sal- 
ary, and time spent on administrative duties. This 
information is collected on the Teacher Service 
Record and includes social security numbers. 

Certification, qualifications, and professional 
development. The ISBE has been collecting data 
on teacher qualifications at least since the 2004/05 
school year, including years of experience (by 
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location), college attended, and highest degree 
obtained (and where). Illinois does not collect data 
on teacher professional development. 

Mobility and attrition. Illinois does not collect 
data on teacher mobility or teacher attrition. 

Linking state data 

The ISBE began assigning a unique student ID 
number in the 2005/06 school year. This ID 
number will be used to link student-level records 
across all state databases. Illinois also began as- 
signing a unique teacher ID number at the same 
time, although the state does not have the ability 
the match teacher records with student records. 



Data quality assurance 

The ISBE has procedures in place to ensure that 
a student (or teacher) is not assigned more than 
one ID number and that two students do not 
receive the same ID number. The state also collects 
student demographic data before administering 
standardized tests and has an audit system for 
checking the accuracy of information submitted. 

Note 

1. See http://www.isbe.net/board/meetings/ 
sept06meeting/fac.pdf for a September 2006 
“Data Warehouse Requirements and Eeasibil- 
ity Study.” 
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APPENDIX C 
INDIANA 



Information needs assessment 

Data collection and quality assurance procedures 

The Division of Educational Information Systems 
(EIS) in the Center for Information Technology is 
responsible for school data and web services and 
the network for IDEAnet, the Official Web site of 
the Indiana Department of Education (see http:// 
www.doe.state.in.us/htmls/divisions.html). Indi- 
ana’s EIS has been described as a service provider 
to the Department of Education, eliminating silos 
that can develop when separate collections occur 
in multiple sub-units. EIS supplies the expertise 
for the development and maintenance of data 
collection systems. The expertise regarding the 
business rules of each data element resides in 
the program area or appropriate division of the 
department. Data are stored in a relational data- 
base (an internally-built XML-enabled collection 
system written in .NET and stored in an Oracle 
database), providing enhanced capacity to cut the 
data multiple ways and produce new reports in 
response to individual offices’ information needs. 

The Division of School Assessment assists schools 
and school corporations in implementing ISTEP-f 
(I ndiana Statewide Testing for Educational 
Progress); assists “educators, parents, and mem- 
bers of the general public in the interpretation of 
individual and group achievement data”; provides 
“technical assistance in matters of psychometrics 
and research”; and works closely with the divi- 
sion of EIS through structured relationships and 
informally (see http://www.doe.state.in.us/assess- 
ment/welcome.html). The Center for Assessment 
and Research is a primary internal departmental 
consumer of Indiana state educational data. Under 
the direction of Assistant Superintendent Wesley 
Bruce, this Center produces a variety of summative 
statistics tracking student educational progress and 
attainment over time, including statistics produced 
in compliance with state and No Child Left Behind 
accountability requirements. The Center maintains 



both informal daily and more structured, formal 
relationships with EIS colleagues to ensure data are 
collected and analyzed appropriately for compli- 
ance and other informational purposes. 

The director of the Division of School Data Report- 
ing in the Office of the Superintendent works with 
local school and district personnel and staff across 
the Indiana Department of Education to ensure 
the responsive, “consistent, timely, accurate, reli- 
able, understandable, and usable” reporting of 
data through a variety of mechanisms, including 
the Accountability System of Academic Progress 
(ASAP) web site, and other online resources and 
media. The director also coordinates the depart- 
ment’s “participation in school assessment data 
research and development projects,” and facilitates 
“the alignment and standardization of school data 
reporting across various media” (see http://www. 
doe.state.in.us/datareporting/welcome.html). 

Data quality is a primary, daily function of EIS. 
The director of School Data Reporting works in an 
oversight role on data quality — often serving as a 
liaison between schools and EIS. Identifying any 
problems that may exist with reported informa- 
tion is a primary role of the Data Collection and 
Reporting team within EIS. At the time of writ- 
ing, two programmer analysts, a senior analyst, 
and a manager were responsible for writing the 
systems and enhancements to the system; an 
additional four individuals were responsible for 
various collection and operations activities (e.g., to 
ensure files transact, that tools are appropriately 
deployed, that data are reported on schedule, that 
data reported but not processed due to problems 
with data layout are resolved). The team also 
provides help-desk/trouble-shooting and other 
services. Regional and local training are provided 
through a training facility available to the state 
education agency and regional education agency 
service centers. Such training is typically provided 
by staff directly responsible for data collections 
(e.g., the Associate Superintendent over all the 
education information systems, or the lead in op- 
erational data collection) as time and opportunity 
allow, rather than by dedicated trainers. 
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With the establishment of the Student Test 
Number (STN) (further to the Indiana General 
Assembly’s Public Law 221), Indiana is able to col- 
lect student-level, longitudinal data.^ The system 
became operational in the 2002/03 school year. 
Currently four years worth of longitudinal data are 
available for select data elements.^ 

Defining state data needs 

How Indiana defines data needs 

Data needs are defined primarily through a com- 
bination of state and federal legislation and regula- 
tions. Indiana is a local control state; the state 
constitution provides that any authority not spe- 
cifically given to the central government reverts to 
the local level — in the case of education, the school 
boards which are the legal entities controlling the 
educational system. Any data reporting beyond 
that required by state law or state board rule is 
voluntary. 

Legislative mandates may specify the granularity 
and frequency with which data must be collected, 
the ways in which it must be coded, the duration 
for which it must be maintained, and/or the meth- 
ods by which it must be analyzed in order to be in 
compliance with various reporting requirements. 
An example is the new state formula for calculat- 
ing graduation rates adopted by the Indiana Gen- 
eral Assembly in 2003, and subsequently codified 
at IC 20-26-13 (see http://www.ai.org/legislative/ic/ 
code/title20/ar26/chl3.html#IC20-26-13-10). The 
new method for calculating high school gradua- 
tion rates capitalized on the state’s ability, through 
the Indiana Student Test Number (STN) system, 
to track individual students’ progress through 
the educational system. Other legislative require- 
ments include provisions of Indiana Public Law 
221 (P.L. 221). Passed two years prior to the federal 
No Child Behind Act of 2001, P.L. 221 established a 
comprehensive accountability system that, among 
other things, places schools “into one of five 
categories based upon ‘improvement’ and ‘perfor- 
mance’ data from the Indiana Statewide Testing 
for Educational Progress-Plus (ISTEP-e)” with 



“category placements for all of the state’s public 
and accredited nonpublic schools” adopted by the 
Indiana State Board of Education for the first time 
in August 2006.^ 

Data needs can be construed as somewhat nar- 
rowly defined — 100 percent of the state’s data is 
collected for required reports and purposes. At the 
same time, the establishment of unit level collec- 
tions creates new opportunities to analyze data 
in new and multiple ways, so the same raw data 
can serve many information needs or aspirations. 
Considerable effort has been expended to ensure 
data can be returned back to schools, school dis- 
tricts, and policy makers as information, including 
through the Indiana Accountability System for 
Academic Progress (ASAP) web site. While Indi- 
ana has a longstanding tradition of making data 
available to users outside the department, previous 
systems for querying elements to extract infor- 
mation from databases required greater under- 
standing of the structure of the data and software 
suitable for performing query and extract func- 
tions. The ASAP system is the result of an effort to 
take advantage of more advanced tools to display 
data in ways more useful to end-users, including a 
question-driven front-end and pre-defined graphi- 
cal and tabular displays. ASAP includes a number 
of tools designed to assist schools in the develop- 
ment of triennial school improvement plans and 
annual reviews and revisions of those plans, and 
to serve anticipated information needs of teachers 
and parents.^ 

Information and data priorities 

The state department of education uses data both 
to comply with legislative reporting requirements 
and to inform decisionmaking at a variety of levels 
in the state educational system. As noted above, 
legislative reporting requirements can specify 
data needs and methods of analysis; they can also 
provide more general guidelines for the collection 
and reporting of information. With the establish- 
ment of unit record systems and student-level data 
collections, a new range of opportunities exists to 
not only use data to inform decisionmaking at the 
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State level but also locally, returning data “back to 
schools.” The EIS also enhances service- delivery 
units’ ability to focus on their core service delivery 
missions, rather than associated data collection 
activities. The more innovative uses of state data 
are a key state education agency priority, and a 
strategic plan has been developed in line with this 
mission to make the department a better customer 
of its data. 

This has implications for the collection, analysis, 
and reporting of information. From one perspec- 
tive, it has been suggested that the EIS will never 
be “finished” but will continue to develop. From 
another vantage point, information “needs” can be 
construed as fluid, flexible, and dynamic; itera- 
tively re-defined to enable anticipatory, proac- 
tive, strategic decisionmaking. Not all the data 
necessary to make maximum use of data already 
collected are reported; for example, student-level 
course-taking data is not currently available. With 
the addition of this information, it would be pos- 
sible to leverage the student-level system Indiana 
has developed to answer additional questions 
about schools’ effectiveness and students’ progress 
through the education system. 

Indiana has developed a strategic plan, articulat- 
ing a medium-term vision to guide development of 
its educational information system. Key informa- 
tion and data priorities are identified in the “State 
Profile” section of the Indiana Accountability Sys- 
tem for Academic Progress (ASAP, online at http:// 
www.doe.state.in.us/asap/welcome.html). 

The remainder of this section reviews the kinds of 
questions the state of Indiana would most like its 
educational data to address in the short-term and 
looking further to the future, and highlights the 
information needs which senior state education 
agency officials currently consider to be particular 
priorities. Existing challenges which may affect 
the state’s ability to achieve these objectives mov- 
ing forward are discussed below. 

Establishing benchmarks. In keeping with the goal 
of leveraging existing data to proactively inform 



decisionmaking, state officials identified several 
issues that could usefully be informed through 
new data analyses, in some cases supplemented or 
enabled by new data collections. The new cohort 
graduation rate, for example, raises questions 
about similarities and differences between stu- 
dents in the cohort of the class of 2006 who gradu- 
ated on time, and those who did not. It has been 
suggested that information about who did not 
graduate could be used to form an Indiana at-risk 
profile, with the potential to identify students in 
seventh and eighth grade who, based on informa- 
tion already available about those who did not 
complete high school on time, may be at risk and 
might benefit from various interventions. Even 
the currently available point information might be 
able to identify key characteristics of students in 
the 2006 cohort who did versus those that did not 
graduate (such as mean age differences, whether 
one is significantly over age for grade, average 
mobility across the groups) that could be used to 
identify potentially at-risk students; in successive 
years, trend data might refine such profiles. 

While there is an interest in being able to compare 
Indiana’s progress with the rest of the nation, dif- 
ferent states often have different definitions and/ 
or operationalizations of key constructs; building 
metrics that would facilitate cross-state compari- 
sons is an interesting but potentially challenging 
task. Of greater interest is the ability to inform and 
evaluate status within the state. Comparisons over 
time are thus particularly meaningful; e.g., are 
fewer students dropping out? Are more students 
completing school on time? Are students well 
prepared for the postsecondary world? Is the state 
doing everything it can to ensure that students re- 
ceive the kinds of supports they require to succeed 
educationally? 

Tracking progress. Access to course completion 
data was identified as a high priority. This infor- 
mation was described as critical to understanding 
the relationship between classes, courses, and 
performances. Currently, with basic demographic 
information and student-level test results there are 
outcome data but essentially no evidence of what 
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treatments have been. With data on which stu- 
dents were enrolled in which courses, and which 
courses were successfully completed by which 
students, as well as information on patterns of ser- 
vices used and electives taken, it maybe possible 
to more effectively identify schools that are doing a 
good job, and schools that may need assistance — 
judgments one is limited in making solely on the 
basis of annual achievement test data. 

With respect to its state accountability system, 
Indiana is considering changing a metric — 
chosen in 1999/2000 to be the “percent of students 
passing”— to scale score growth. Such a change 
would be consistent with P.L. 221, which includes 
both a status and a growth piece. The performance 
of all students sets a starting point, although the 
system expectation is that schools are only ac- 
countable for those students they actually served 
for most of the school year. With the ability to 
follow students longitudinally, it should be possible 
to identify (and acknowledge) schools on the basis 
of their ability to move students relative to previous 
performance. So, for example, effectiveness might 
include moving all students in a school upward on 
a distribution, acknowledging that educationally 
a student who moves from the bottom quartile to 
the next quartile is learning more than expected in 
a single year — a significant achievement, even if it 
does not take the student to proficiency. 

As part of its interest in enabling schools and 
districts to leverage data collected at the state level, 
Indiana is interested to establish and has submit- 
ted a request for funding for a data warehouse. An 
underlying consideration is the enhanced ability 
for the state to provide data back to the schools. 
One option would be to provide schools with 
direct access to data in a real-time (e.g., web- 
enabled) manner, allowing access through tiered 
security consistent with FERPA. Funding (notably 
for additional staff resources) is a key challenge 
to establishing such a data warehouse. Indiana 
is a local control state, with over 20 versions of 
student information systems being used by local 
entities. Additional resources would enable the 
state to build on the XML infrastructure already 



established,® such as creating data dictionaries to 
make connections between individual databases. 

The resource considerations associated with the 
move to a statewide longitudinal data system are 
considerable. Whereas previously the aggregate 
data collection system was collecting thousands of 
pieces of information, the state is now collecting 
millions of pieces of information; however it has 
gone to the student level with the same resources it 
used previously to collect aggregate data. Obtain- 
ing resources to build out the system is a signifi- 
cant endeavor as well as a priority. Additional 
resources would be required to realize another 
potential goal — establishing a direct connection 
between the current K-12 and the postsecondary 
education data systems. 

Information on teacher professional development 
is another priority. The current teacher licensure 
system requires ongoing professional develop- 
ment. While the state provides approximately $13 
million in funds for professional development, 
there is no electronic tie to specify what develop- 
ment individual teachers receive from those funds. 
Many of the core elements necessary to such a 
system are already available (such as licensing 
information, a database of teachers connected to 
schools), but there is not currently a connection 
for teachers as users to sort and review their own 
individual employment histories. 

Commenting on the priority attached to obtaining 
information for curriculum development, a state ed- 
ucation agency official underscored the importance 
of collecting course information for students — 
including the desirability of providing a mechanism 
to directly link performance on tests (SAT, AP, and 
state tests) with courses that students have taken. A 
major challenge to making this information avail- 
able is the associated (additional) burden in such 
fairly complex data collection efforts. The ability to 
link to post secondary education data was described 
as another priority for Indiana. Key considerations 
here include a common format across record-keep- 
ing systems, access to course completion informa- 
tion, and a suitable identifier. 
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Understanding state data challenges 

Resolving issues associated with the burden data 
collections can place on local school districts was 
identified as perhaps the biggest challenge for the 
state of Indiana; the state can program data collec- 
tions, but this creates a burden on local districts 
to deliver on those collections — and with the 
move from collecting aggregate to student-level 
data, this is acknowledged to require a significant 
effort at the local level. For this reason the state 
has worked to phase in the roll-out of new col- 
lections over time, providing schools with six 
months to one year’s notice of changes to existing 
or the implementation of new collections so that 
they have adequate time to make modifications to 
their data systems. Financial resources also are a 
consideration in this regard, as the state provides 
an architecture in which data is to be delivered to 
the state rather than a common data system. The 
state does not have additional funds to provide 
help to modify such collection systems, although 
local units can work with a 13-member advisory 
committee consisting of various personnel from 
school corporations to design reports and stream- 
line data collection. Such resource considerations 
are anticipated to be important in planning for the 
collection of student-level course completion data. 

As noted above, currently the state of Indiana has 
only aggregate data about student course informa- 
tion, and no information on course completion. 
Collecting course taking data at a student level is 
expected to be a substantial undertaking, involv- 
ing not only the collection of student-level data but 
also the ability to connect student and teacher data 
(for example to monitor outcomes associated with 
highly qualified teachers), and requiring the devel- 
opment of mechanisms to combine information on 
courses delivered on a quarter system, a trimester 
system, and a traditional semester system — all 
systems that exist in the state of Indiana. 

Indiana is not experiencing any difficulties 
complying with federal reporting requirements, 
although the commitment of resources necessary 
to satisfy these requirements was mentioned. It 



was estimated that significantly less than one- 
tenth of an FTE (0.02) might previously have been 
devoted to reporting under CCD (in part because 
aggregate data was collected by the state in a way 
that was easy to report). Currently the state has 
one full-time programmer dedicated to EDEN, 
and that person requires supplemental time from 
others of no less than 0.5 FTE. Such reallocations 
limit the staff capacity available to move forward 
with additional enhancements to the state’s 
student-level data collections. The availability of 
additional resources to enhance student-level data 
systems (beyond the limited number of grants 
available to support statewide longitudinal data 
systems— possibly to include funds the state could 
pass on to local entities moving from aggregate, 
essentially semi-paper electronic collections of 
paper documents to enter information directly 
into a relational database) would be welcomed. 

Inconsistencies in data definitions over time were 
felt to pose no more than small, typically insignifi- 
cant challenges for the state of Indiana. In Novem- 
ber 2006 the state was awarded an NCES Special 
Task Order for the development of a data diction- 
ary; this project is expected to begin in February 
or March 2007. It was noted that as the system 
evolves additional data is periodically collected, 
and that in such instances it may be difficult (or 
impossible) to conduct certain analyses retrospec- 
tively. As an example, Indiana previously collected 
annual mobility data on students at the end of the 
school year, but not the date at which students 
moved. With the move to the calculation of cohort 
graduation rates, a question arose regarding how 
to address instances of students who were both 
mobile and dropouts in the same year; (nine such 
instances arose out of a 2006 graduation cohort 
of approximately 79,000). As a result, a date field 
has been added to every mobility record. Such 
new data is extremely helpful moving forward, but 
limits the kinds of comparisons that can be made 
looking backward. Similarly, inconsistencies in 
data definitions across data sets have proved chal- 
lenging in the past, although these were observed 
to be typically associated with the prevalence of 
multiple definitions outside of rather than within 
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the state. Within the state, Indiana has built its 
current data collection system from the ground 
up, so is not troubled with such differences in data 
definitions internally. 

Inconsistencies in coding procedures (across 
items, data sets, and/or over time) or the intervals 
between data collections are not issues for the state 
of Indiana. Inconsistencies in the treatment of 
missing data were described as more problematic, 
particularly in the startup of the student-level data 
collection system. For example, in the first year 
of collection, information on the student’s prior 
grade level was not available; as a result, it was 
not possible to take advantage of capacity in the 
system to set flags to check student retention status. 
Some such ‘start-up’ issues persisted beyond year 
one, as data collections have been phased-in, not 
rolled-out simultaneously. Steps taken to address 
the challenge of defining and coding missing data 
include creating special reports listing-out students 
with missing (or inconsistent, or incomplete) data 
for schools or districts to complete. Reasonability 
checks are also built into the system so that, for 
example, it is not possible for schools to submit 
conflicting attendance reports. Other reasonability 
checks are longitudinal (precluding the recording 
of a student as ‘graduating’ two years in a row). On 
balance, the system is ‘smarter’ and continues to 
evolve in ways that minimize challenges associated 
with missing data. The only resource identified as 
potentially beneficial to address remaining issues 
was additional staff resources. Indiana’s student- 
level data collection system has been built with 
existing staff (such as the state has moved from an 
aggregate to a student-level system with the same 
staffing). Additional funds have been requested 
from the legislature (to enable a move to a state- 
level data warehouse); it is an open issue whether or 
not these resources will be forthcoming. 

Data entry errors, while described as fairly isolat- 
ed — (infrequent, in the context of the hundreds of 
millions of records passed every year) — are still 
characterized as problematic in as much as any 
errors are unacceptable. To help ameliorate such 
difficulties, the state describes specific roles and 



responsibilities with respect to student data col- 
lection (for principals, teachers, staff responsible 
for data entry, and central office staff), including 
how to look for such errors and ensure the data 
submitted are as complete as possible. Despite 
these efforts some problems remain, in part due to 
issues of transition in schools (e.g., staff turnover, 
described as significant, with approximately 28 
percent of the positions directly responsible for the 
statewide test at the district level occupied by dif- 
ferent individuals than the previous year). 

Recognizing and addressing these issues requires 
a system with more support built into it, and more 
‘help desk’ functionality. Indiana has established 
an advisory committee of local individuals who 
help both design and roll-out collections, includ- 
ing piloting all collections. The state works to 
provide easily accessible roles and responsibili- 
ties documents, and to provide reports that look 
similar to older aggregate reports, providing a 
high-level view of data in a more familiar format 
to those who work to ensure the validity of data 
submitted. In part as a result of initiatives such as 
these, overall, data entry issues have not impaired 
Indiana’s ability to use data resources in an ef- 
fective manner. On a ‘wish list’ for the future is 
the possibility of putting together a ‘dashboard’ 
to provide individuals responsible for approving 
data with an overview (e.g., of what data has been 
reported, when it was reported, what it looks like) 
without requiring them to deal with the opera- 
tional system used to submit data, or to under- 
stand how the data system itself ‘works.’ 

Staff capacity presents a potential challenge to 
speedy responses from the state education agency 
to requests for information; currently there is no 
committed resource to respond to data requests and 
extra staff capacity is not a characteristic of the sys- 
tem, so new requests are likely to involve reassign- 
ment of staff to write and quality-control the queries 
necessary to extract data. Steps taken to address 
such challenges include applying for relevant grants 
and attempting to anticipate additional information 
needs to produce proactively a full range of reports 
with the greatest possible functionality. 



24 



GETTING THE EVIDENCE FOR EVIDENCE-BASED INITIATIVES 



State data inventory 

Indiana's educational data system 

The 2006 strategic plan describes seven core values 
and five ‘strategic goal areas’ for the Indiana De- 
partment of Education. ® Goal IV, “the Department 
of Education will have the information and pro- 
cesses to provide timely, accurate, and professional 
service,” aligns closely with the core value state- 
ment that the Indiana Department of Education 
“must make policy and program decisions based on 
timely, accurate data” (Indiana Department of Edu- 
cation, 2006: 2 & 6). Three objectives are articulated 
under Goal IV; under each objective specific action 
items are identified as follows (see table Cl). 

Table C2 identifies key contacts available to 
provide additional information on Indiana’s data 
processes and files. Included in this table are the 
Objective Sponsors for the three objectives falling 
under Goal IV of the 2006 strategic plan.^ 

The Indiana Accountability System for Academic 
Progress (ASAP) web site provides a portal for 
accessing: (1) information on academic standards, 
accountability, accreditation, best practices, pro- 
fessional development, and school improvement, 
(2) school data, and (3) state profile information. 
Among other things ASAP’s academic standards 
pages provide information on the Indiana aca- 
demic standards and resources for courses in 



advanced life science, agriculture education, busi- 
ness and marketing, dance. English/language arts, 
family and consumer sciences, health, mathemat- 
ics, music, physical education, science, social stud- 
ies, technology education, theatre, visual arts, and 
world language (see http://www.doe.state.in.us/ 
asap/academicstandards.html). The accountability 
pages include: information on assessments that are 
required as primary indicators of improvement 
and performance, and used to monitor student 
success; the definition of school performance and 
improvement under P.L. 221; and the identification 
of students who are eligible for inclusion in the 
non-mobile cohort whose achievement is tracked 
in school improvement indicators (see http://www. 
doe.state.in.us/asap/accountability.html). The ac- 
creditation pages describe the “quality assurance 
process that confirms that an Indiana school is 
meeting minimum standards in student perfor- 
mance, legal standards, and school improvement 
planning” (see http://www.doe.state.in.us/asap/ 
accreditation.html). The best practice page pro- 
vides links to “current research-based strategies 
involving a variety of areas within the learning 
environment” (see http://www.doe.state.in.us/ 
asap/bestpractice.html). 

Accessible from the professional development 
page are information on the requirements of a 
professional development program (a “required 
component of the school improvement plan”), the 
core principles for professional development, and 



TABLE Cl 

Goal IV objectives and action items 

Objective 1: Improve the availability of information to employees 

"Centralize and share internal information in a simple and effective way." 

"Create an internal program to promote a culture that values information sharing." 

Objective 2: Improve the quality and availability of information to constituencies 
"Create a centralized support center to respond to external inquiries and visits." 

"Implement outreach activities to increase general public's knowledge of critical instructional areas." 

Objective 3: Improve the quality of services to constituencies 
"Create an inventory of DOE services to constituencies." 

"Align services provided to constituencies with the DOE's Vision and Mission." 

"Develop and implement a plan to improve services to schools based on an assessment of serviced utilization and outcomes." 

Source: Indiana Department of Education Strategic Plan, September 2006 (online at http://www.doe.state.in.us/strategicplan/docs/IDOE-StrategicPlan- 
Complete.pdf); see p. 17. 
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TABLE C2 

Key contacts for more information on Indiana's data processes and files 



1 These activities . . . 


... are currently the responsibility of: I 


Center for Assessment and Research 


Wesley D. Bruce 

Assistant Superintendent 

Center for Assessment and Research 

Phone: (317) 232 9050 

E-mail: wbruce@doe.state.in.us 


Educational Information Systems 


Anne Brinson 

Associate Superintendent 

Division of Educational Information Systems 

Phone: (317) 232 0808 

E-mail: abrinson@doe.state.in.us 


School assessment 

The Division of School Assessment "assists indiana school corporations 
and schools in the implementation of the Indiana Statewide Testing for 
Educational Progress (ISTEP+)," assists "educators, parents, and members of 
the general public in the interpretation of individual and group achievement 
data," and provides "technical assistance in matters of psychometrics and 
research" (see http://www.doe.state.in.us/assessment/welcome.html) 


Michele Walker 

Director, Division of School Assessment 
Phone: (317) 232 9050 


School Data Reporting 

The Director of School Data Reporting works with staff across the Indiana 
Department of Education and with local school district & building personnel 
to ensure data are consistent, timely, accurate, reliable, understandable, & 
usable; and coordinates Departmental "participation in school assessment 
data research and development projects" (see http://www.doe.state.in.us/ 
datareporting/welcome.html) 


Mary Tiede Wilhelmus 
Director of School Data Reporting 
Phone: (317) 232 9177 
E-mail: mtw@doe.state.in.us 


Strategic plan Goal IV objectives 1 & 2 

"Improve the availability of information to employees" 

"Improve the quality and availability of information to constituencies" (see 
pages 6 and 7 of the Strategic Plan online at http://www.doe.state.in.us/ 
strategicpIan/docs/IDOE-StrategicPIan-Complete.pdf) 


Kevin McDowell 

Office of the Superintendent 

Phone: (317) 232 6676 


Strategic plan Goal IV objective 3 

"Improve the quality of services to constituencies" (see pages 6 and 7 of the 
Strategic Plan online at http://www.doe.state.in.us/strategicplan/docs/IDOE- 
StrategicPlan-Complete.pdf) 


Robert Marra 
Phone: (317) 232 0570 



criteria for approving the professional development 
program (see http://www.doe.state.in.us/asap/ 
prodev.html). From the school improvement plan 
home page information is available on required 
and optional components of an Indiana school 
improvement plan, and information that can assist 
schools in their improvement planning processes 
(see http://www.doe.state.in.us/asap/sip.html). 



state.in.us/asap/data.html). The state profile sec- 
tion provides “information about state level demo- 
graphics, assessment results and Indiana’s latest 
initiatives that impact education” (see http://www. 
doe.state.in.us/asap/welcome.html and http:// 
www.doe.state.in.us/asap/stateprofile.html). 

Key data elements 



The school data section of the ASAP web site 
“provides informational, demographic, and 
achievement data about Indiana schools” that can 
be disaggregated and graphically displayed, and 
compared to similar schools (see http://www.doe. 



Student data 

Demographics. A variety of student-level demo- 
graphic information has been collected since 
2005/06, including gender, date of birth, race and 





26 



GETTING THE EVIDENCE FOR EVIDENCE-BASED INITIATIVES 



ethnicity. Data on free/reduced lunch eligibility has 
been collected annually at the student level since 
2002/03. Key demographic information is submitted 
to the Indiana Department of Education in the STN 
Lookup Data (DOE-STN) data collection (a “required 
collection for all public schools, accredited nonpub- 
lic school, charter schools and freeway schools”) in 
order to “populate the Student Test Number (STN) 
Lookup System.”® Included here are student gender, 
birth date, and race/ethnicity; all are required fields. 
Student socioeconomic status (SES) is recorded in 
the annual Pupil Enrollment report (DOE-PE) ac- 
cording to whether or not the student is approved for 
free or reduced lunch. This field is coded l=student 
approved for free meals/milk, 2=student approved 
for reduced price meals, or 3=student not approved 
for free or reduced meals/milk (see DOE-PE, Version 
07.24.06, http://www.doe.state.in.us/stn/pdf/2006-07 
-24-pupil-enrollment.pdf).® The Additional Student 
Information (DOE-AD) report collects specific 
information required to “complete the information 
on student groups” which is “used for federal and 
state reporting” included here is a ‘Homeless’ field 
used to record whether individual students were 
“considered homeless at any time during the school 
year while attending this school” (see DOE-AD, 
Version 12.14.06, http://www.doe.state.in.us/stn/ 
pdf/2006-12-14-ad.pdf). Students are considered 
homeless “if they lack a fixed, regular, adequate, 
nighttime residence.” ‘Homeless’ is a required field 
in the DOE-AD report. 

A variety of demographic information is recorded 
in the Language Minority report (DOE -EM), 
which gathers “student information on language 
minority and immigrant students”; fields of inter- 
est include ‘Language Minority Status’, ‘Non-U.S. 
Origin (Immigrant)’, ‘Country of Origin’, and 
‘Native Language Code’ (see http://www.doe.state. 
in.us/stn/pdf/2006-10-03-lm.pdf).^° 

Enrollment. Another field included in the STN 
Lookup Data collection is the state assigned school 
ID (school number) indicating the “school number 
where the student is located or last known school.” 
These files are scheduled for submission on or near 
the 15th of each month, with records “including 



updated student information and/or information 
for students who are entering the educational 
experience for the first time,” (see http://www.doe. 
state.in.us/stn/pdf/2006-05-30-stn-lookup-data. 
pdf). More detailed enrollment information is cap- 
tured in the Pupil Enrollment Report (DOE-PE), a 
collection required for public schools, accredited 
nonpublic schools, charter schools, and freeway 
schools in order to provide student enrollment 
information as of October 1st each school year. 

The student census obtained in this data collec- 
tion is used “for disbursement of state grants and 
Adequate Yearly Performance (AYP).” Six required 
fields are reported during this data collection: 
school number; student test number (STN); grade 
level;^^ socio-economic status; language minority 
status; and ‘retained’ — a field recording whether or 
not the student was retained in the same grade as 
the previous year (see DOE-PE, Version 07.24.06, 
http://www.doe.state.in.us/stn/pdf/2006-07-24- 
pupil-enrollment.pdf). Also useful here is informa- 
tion contained in the Dropout and Mobility Report 
(DOE-DM); this report is discussed in detail in the 
section on Graduation and Dropout data, below. 

Course-taking. A list of subjects (and levels) taught in 
the 2006/07 school year is available in the “Subject 
and Level Code List for DOE-CP” (see http://doe. 
state.in.us/stn/pdf/cp_codebook.pdf). Individual 
student-level course-taking and course-completion 
data are not currently collected at the state level. 

Special program participation. The Programs and 
Services report (DOE-PS), a required data collection 
for all students served by public schools, accredited 
nonpublic schools, charter schools, and freeway 
schools, was designed “to gather program and ser- 
vices information that students receive throughout 
the school year” for federal and state reporting; this 
report was retired in the 2005-06 data collection 
year (see DOE-PS, Version 12.01.05, http://www. 
doe.state.in.us/stn/pdf/2005-12-01-ProgServices. 
pdf). Key fields in that report included: ‘Special 
Education Participant’ which indicates whether or 
not students have an individualized education plan 
(lEP); for special education participants two fields 
indicating the student’s primary and secondary 
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eligibility classifications (‘Primary Exceptionality’ 
and ‘Secondary Exceptionality’); ‘Special Education 
Placement Type’ to indicate the type of program 
in which special education students participate; 
‘Special Education Referral’ to record whether or 
not students were referred for special education 
evaluation but not placed in a program; ‘Section 504 
Plan’ to indicate students served by such plans; and 
‘ESEA Title I Participant’ to record whether or not 
students were Title I participants at any time during 
the pertinent school year. Effective December 1, 
2007, Special Education data are collected via the 
DOE-SE (http://www.doe.state.in.us/stn/pdf/2006- 
11-14-se.pdf). ‘Gifted and Talented’ data are col- 
lected via the Additional Student Information data 
collection (i.e., DOE -AD). 

The Title I (DOE-TI) report is designed “to gather 
Title I services a student receives, or is eligible to 
receive, throughout the school year” in the fall, 
spring, and summer in “public, accredited non- 
public schools, charter schools, and neglected in- 
stitutions that have Title I funding,” (see DOE-TI, 
Version 08.14.06, http://www.doe.state.in.us/stn/ 
pdf/titlel.pdf). Relevant fields include ‘ESEA Title 
I Program’ used to record which Title Program 
is providing services to a student; and a range 
of fields used to record whether or not students 
receive Title I instruction in particular areas. 

The annual Pupil Enrollment collection (DOE-PE) 
reports information on student language minority 
status (also referred to as limited English profi- 
cient, LEP, ESL, or ELL). Two proficiency codes are 
allowable to describe language minority students: 
l=fluent English proficient (FEP), and 2=limited 
English proficient (LEP), with the field left blank for 
those who are not language minority students (see 
DOE-PE, Version 07.24.06, http://www.doe.state. 
in.us/stn/pdf/2006-07-24-pupil-enrollment.pdf). 

Information on students receiving Special Educa- 
tion services and the services that they receive is 
reported in the Special Education (DOE-SE) data 
collection. A required collection for both public 
and charter schools, DOE-SE data are reported in 
October and April for informational purposes, and 



in December to be reconciled with “CODA data to 
determine final counts for funding.” Information 
reported in this data collection include: school 
number; corporation of legal settlement; student 
test number; grade level; primary and secondary 
exceptionality;^^ special education placement type; 
socioeconomic status; the type of facility provid- 
ing special education services when that facility is 
not a public school, nonpublic school, or charter 
school; and the service site state assigned number 
“for Neglected Institutions and Health Facilities 
where the student received Special Education ser- 
vices” (see DOE-SE Version 11.14.06, http://www. 
doe.state.in.us/stn/pdf/2006-ll-14-se.pdf). 

The LEP/ISTEP-(- report (DOE-LEPl) is used to 
“gather student information on all first year LEP 
[limited English proficient] students enrolled in 
the school corporations implementing the ISTEP-t- 
Flexibility for first year LEP students” (see DOE- 
LEPl, Version 10.10.06). As detailed there, “School 
corporations are required to administer ISTEP-t- 
math and science assessments to LEP students who 
have been enrolled in U.S. schools for less than one 
year. . .”; the LEP/ISTEP-r records, for “first year 
limited English proficient (LEP) students exempted 
from English/Language Arts portion of ISTEP-i-” 
the date the student was first enrolled in U.S. school 
(see DOE-LEPl, Version 10.10.06, http://www.doe. 
state.in.us/stn/pdf/2006-10-10-lep-istep.pdf). 

The Language Minority report (DOE-LM) re- 
quires public schools, accredited nonpublic 
schools, charter schools and freeway schools 
to report a range of data on language minority 
and immigrant students including in the field 
“Instructional Program” the “English Language 
education instructional program” providing 
service to individual students. The DOE-LM is 
also used to record, for language minority and/or 
immigrant students whether individual students 
are “officially considered to be participating in a 
Special Education Program” (in the field ‘Special 
education Participant’); whether or not students 
“participate in a Gifted and Talented program” (in 
the ‘Gifted & Talented Participant’ field); and the 
Title 1 program providing service to the students 
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(in the field ‘ESEA Title I Program’) (see DOE-LM, 
Version 10.03.06, http://www.doe.state.in.us/stn/ 
pdf/2006-10-03-lm.pdf). 

The Additional Student Information (DOE-AD) 
report collects specific information required to 
“complete the information on student groups” 
which is “used for federal and state reporting”; 
included here is a required field, ‘High Ability 
(G/T),’ used to record whether individual students 
participating “in the Gifted and Talented pro- 
gram show high ability” in one of four areas (see 
DOE-AD, Version 12.14.06, http://www.doe. state. 
in.us/stn/pdf/2006-12-14-ad.pdf). 

The Homebound/Hospitalized report (DOE-HB) is 
a “required collection for public, accredited non- 
public schools, charter schools and freeway schools” 
designed to “gather information on students that 
receive instruction at home or at a hospital due to 
injury or illness” at any time during a specified col- 
lection period “for federal and state reporting” (see 
DOE-HB, Version 05.30.06, http://www.doe.state. 
in.us/stn/pdf/2006-05-30-homebound.pdf). 

Achievement. Student-level test data has been col- 
lected annually since 2002-03. Relevant data sources 
and instrumentation, rubrics and revisions are 
available online.^^ Information is also collected and 
maintained on untested students in tested grades, 
although not the reason why students were not 
tested. Current policy provides that student-level 
data should be destroyed after five years (see http:// 
www.doe.state.in.us/istep/welcome.html). Another 
indication of achievement is the ‘Type of Graduate” 
data recorded on the Graduate Report (DOE-GR) 
for graduating students receiving diplomas or other 
documents (e.g., Indiana Academic Honors Diplo- 
mas, Core 40 Diplomas, and Certificates of Achieve- 
ment provided to Special Education Students “not 
capable of earning a diploma, but who completed the 
public school educational program prescribed in the 
student’s lEP,” see DOE-GR Version 06.23.06, http:// 
www.doe.state.in.us/stn/pdf/doe_gr.pdf). Also of 
interest here may be the ‘Post Graduate Information’ 
field on the DOE-GR, a required field to indicate 
the type of higher education (if any) each student 



is pursuing as of the autumn following graduation 
(October 2006 for the 2005/06 academic year). 

English language proficiency data is also col- 
lected for select students. Specifically, the Lan- 
guage Minority report (DOE-LM) requires public 
schools, accredited nonpublic schools, charter 
schools and freeway schools to report a range 
of data on language minority and immigrant 
students, including English proficiency-level data 
from testing done in the current school year. Key 
fields here include ‘Language Minority Status’ and 
‘English Language Proficiency Level.’^'* Recorded 
in the ‘Instrument Used’ field is “the instrument 
used to measure the English proficiency level” (see 
DOE-LM, Version 10.03.06, http://www.doe.state. 
in.us/stn/pdf/2006-10-03-lm.pdf). 

Graduation and dropout data. The Graduate 
Report (DOE-GR), a “required collection for public 
schools, accredited non-public schools and charter 
schools that have students in grade 12” counts 
pupils the reporting school “issued a diploma or a 
document ... in the prior school year regardless 
of the pupil’s resident corporation.” The ‘Type of 
Graduate’ field on this report records the type of 
diploma or document each graduating student re- 
ceived (see DOE-GR Version 06.23.06, http://www. 
doe.state.in.us/stn/pdf/doe_gr.pdf). With respect 
to dropout data, the Dropout and Mobility Report 
(DOE-DM), a “required collection for students 
in grades 7-12 that are being educated in public 
schools, accredited non public schools, charter 
schools, and freeway schools,” includes a ‘Drop- 
out or Mobility Code’ used to record the primary 
reason for student dropout or student mobility (see 
DOE-DM, Version 09.14.06, http://www.doe.state. 
in.us/stn/pdf/doe_dm.pdf). For students who drop- 
out, the DOE-DM report also records the ‘Dropout 
Date’ and the student’s last known address (all 
required fields for those who drop out). 

Teacher data 

Demographics. The Certified Employee data collec- 
tion is designed “to gather certified employee in- 
formation” for those “employed as of October 1 of 
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the academic school year” (see Certified Employee, 
Version 10.06.06, http://www.doe.state.in.us/stn/ 
pdfZ2006-10-06-ce.pdf). A variety of demographic 
information is reported in this data collection; 
fields of interest include ‘Ethnicity,’ ‘Gender,’ and 
‘Birth Date’. 



corporation during the period of July 1 to June 
30”) collected in the Certified Employee report 
(see Version 10.06.06, http://www.doe.state.in.us/ 
stn/pdf/2006-10-06-ce.pdf). 

Linking state data 



Certification, qualifications, and professional de- 
velopment. As noted above, the Certified Employee 
report collects information on those employed as 
of the first of October for the academic school year 
(see Certified Employee, Version 10.06.06, http:// 
www.doe.state.in.us/stn/pdf/2006-10-06-ce.pdf). 

Relevant fields reported in the Certified Employee 
collection include: ‘Degree’; ‘Total Years Experi- 
ence’; ‘First Year Teacher’; ‘Prior Year Employment’ 

(with 12 allowable codes indicating, among other 
things, whether individuals were employed in 
schools of the reporting corporation in certified 
or non-certified positions, in other Indiana public 
school corporations, in a public school outside the 
state of Indiana, in a college or university, in busi- 
ness or industry, in the military or governmental 
service, as a homemaker, or were attending a col- 
lege or university in or out of state); ‘Total Percent 
of Time Employed’ (in the reporting corporation); 
whether an individual serves as a ‘Special Popula- 
tions Employee’; and ‘Highly Qualified Teacher’ Notes 
to indicate whether an employee has been “deter- 
mined as a highly qualified teacher in a position 
during this school year.” A companion data collec- 
tion, the Certified Positions report, is “a required 
collection for public schools, accredited non-public 
schools and charter schools” that gathers “the 
positions on certified employees as of October 1 of 
the academic school year” (see Certified Positions, 

Version 10.12.06, http://www.doe.state.in.us/stn/ 
pdfZ2006-10-12-cp.pdf). Relevant fields recorded 
in the Certified Positions collection include: ‘Sub- 
ject’ and ‘Level’ taught; ‘Periods per Week’; ‘Num- 
ber of Pupils’; and, for special education teachers, 

‘Special Education Classroom Settings.’ 

Mobility and attrition. Teacher mobility and at- 
trition data are not currently collected, although 
of potential interest here is the field ‘Contract 
Days’ (“number of days employed in this school 



Indiana’s Student Test Number (STN) is employed 
in the collection of a variety of student-level data 
with potential for the ability to link data across 
collections (see http://www.doe.state.in.us/stn/ 
welcome.html). 

Data quaiity assurance 

Numerous quality control checks are embedded 
in Indiana Department of Education data collec- 
tion activities. For example, should two or more 
schools claim the same student in the annual 
Pupil Enrollment collection (DOE-PE), the result- 
ing conflict must be resolved before the end of the 
collection period.'^ Similarly, conflicts in reports 
on students receiving Special Education services 
are to be settled by the reporting corporations (see 
DOE-SE, Version 11.14.06, http://www.doe.state. 
in.us/stn/pdf/2006-ll-14-se.pdf). 



1. As described in the March 2002 edition of the 
Indiana Department of Education’s Student 
Test Number News (available online at http:// 
www.doe.state.in.us/stn/pdf/STNnews_mar. 
pdf), the student test number (STN) require- 
ment “was created by the Indiana General 
Assembly through Public Law 221 and, 
specifically, says: ‘the department shall assess 
improvement in the following manner: Com- 
pare the results for a school by comparing 
each student’s results for each grade with the 
student’s prior year results. . .” For additional 
information on the STN, see the Department 
of Education’s STN web site, at http://www. 
doe.state.in.us/stn/welcome.html. 

2. Additional reporting requirements have been 
added over time. 
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3. Indiana Department of Education Public Law 
221 (P.L. 221) Fact Sheet, (http://www.doe. 
state.in.us/pl221/docs/PL221%20Fact%20 
Sheet-8-9-06.pdf). For additional information 
see the P.L. 221 home page on IDEAnet, the 
Official Web site of the Indiana Department 
of Education (at http://www.doe.state.in.us/ 
pl221/welcome.html). 

4. See http://www.doe.state.in.us/asap/welcome. 
html. 

5. XML is supported, although the majority of 
schools submit data in CSV format. 

6. The strategic plan, released by Superintendent 
of Public Instruction Suellen Reed in October 
2006, “offers nearly 60 recommendations for 
aligning, guiding, strengthening and improv- 
ing the agency’s efficiency and performance,” 
(see “Reed Unveils Agency-Wide Strategic 
Plan: Outlines Vision for Better Supporting 
Indiana Students and Schools,” IDOL News 
Release http://www.doe.state.in.us/reed/ 
newsr/2006/10-0ctober/strategic-plan.html). 
The 2006 Indiana Department of Education 
Strategic Plan is available online at http:// 
www.doe.state.in.us/strategicplan/docs/IDOE- 
StrategicPlan-Complete.pdf . 

7. As described in the Indiana Department of 
Education strategic plan, the Objective Spon- 
sor assumes a project-specific, “high-level, 
ad-hoc leadership role” with responsibility 
for: “defining the project’s key objectives 
and constraints; securing the commitment 
of human and financial resources; securing 
commitment of key constituents; providing 
EMC [Indiana Department of Education Ex- 
ecutive Management Council] -level guidance 
and direction to the project team; remov- 
ing barriers to success; approving changes 

to scope, cost, and duration; approving and 
signing off on all deliverables; champion- 
ing the project internally and externally, as 
required; monitoring and reporting progress 
on project goals and metrics; and developing 



implementation timelines for each strategic 
objective,” (p. 12). 

8. This data is entered through the secure STN 
Application Center; see STN Lookup Data 
(DOE-STN), Version 05.30.06; retrieved 
online from http://www.doe.state.in.us/stn/ 
pdf/2006-05-30-stn-lookup-data.pdf,18 Dec 
2006. 

9. The same codes were allowed for the ‘Socio- 
economic Status (SES)’ field on the Programs 
and Services report (DOE-PS); (see DOE-PS, 
Version 12.01.05, http://www.doe.state.in.us/ 
stn/pdf/2005-12-01-ProgServices.pdf). SES is 
also reported in the quarterly Special Educa- 
tion (DOE-SE) data collection; see DOE-SE 
Version 11.14.06, http://www.doe.state.in.us/ 
stn/pdf/2006-ll-14-se.pdf 

10. ‘Language Minority Status’ is also a field on 
the Title I report, as is ‘Migrant Student’ sta- 
tus (see DOE-TI, Version 08.14.06, http://www. 
doe.state.in.us.stn.pdf/titlel.pdf). 

11. Grade level is also reported in other collec- 
tions; e.g., the Additional Student Informa- 
tion (DOE -AD) report (see DOE -AD, Version 
12.14.06, http://www.doe.state.in.us/stn/ 
pdfZ2006-12-14-ad.pdf); the Homebound/ 
Hospitalized report (see DOE-HB, Ver- 
sion 05.30.06, http://www.doe.state.in.us/ 
stn/pdf/2006-05-30-homebound.pdf); the 
Language Minority (DOE-LM) report (see 
DOE-LM, Version 10.03.06, http://www. 
doe.state.in.us/stn/pdf/2006-10-03-lm.pdf); 
the Programs and Services (DOE-PS) re- 
port (see DOE-PS, Version 12.01.05, http:// 
www.doe.state.in.us/stn/pdf/2005-12-01- 
ProgServices.pdf); the quarterly Special 
Education (DOE-SE) data collection (see 
DOE-SE Version 11.14.06, http://www.doe. 
state.in.us/stn/pdf/2006-ll-14-se.pdf0; and 
the Dropout and Mobility Report. As the 
latter collection is only required for students 
in grades 7 through 12, allowable codes for 
the ‘Grade Level’ field in the Dropout and 
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Mobility report are 07 through 13 (where 
13=Grade 12+/Adult); (see DOE-DM, Version 
09.14.06, http://www.doe.state.in.us/stn/pdf/ 
doe_dm.pdf). 

12. Allowable codes include: 01=multiple dis- 
abilities, 02=orthopedic impairment, 
03=visual impairment, 04=hearing impair- 
ment, 05=emotional disability (full time), 
06=emotional disability (other), 07=learning 
disability, 08=developmental delay (ages 
3-5A only), 09=communication disorder, 
10=mild mental disability, ll=moderate 
mental disability, 12=severe mental disability, 
14=deaf-blind, 15=autism spectrum disorder, 
16=traumatic brain injury, 17=other health 
impairment, and for secondary exceptionality 
99=no secondary exceptionality; see DOE-SE 
Version 11.14.06, http://www.doe.state.in.us/ 
stn/pdf/2006-ll-14-se.pdf 



13. See http://www.doe.state.in.us/istep/Program- 
Manual.html and http://www.doe.state.in.us/ 
istep/publications.html. Rubrics and revisions 
are available at http://www.doe.state.in.us/ 
istep/2002/02grd8_ruhricmemo.html. 

14. ‘Language Minority Status’ is also a re- 
quired field on the Pupil Enrollment Report 
(DOE-PE) (see DOE-PE, Version 07.24.06, 
http://www.doe.state.in.us.stn/pdf/2006-07- 
24-pupil-enrollment.pdf). 

15. As described in DOE-PE, version 07.24.06, in 
such cases “it is up to the schools to decide 
which school will report the pupil. If two or 
more schools report the same pupil then the 
pupil will be excluded from the enrollment 
count of all schools until the pupil is reported 
by a single school,” (http://www.doe.state.in.us/ 
stn/pdf/2006-07-24-pupil-enrollment.pdf). 
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Information needs assessment 

Data collection and quality assurance procedures 

Student information is gathered using the Elec- 
tronic Access System for Iowa Education Records 
(EASIER). Multiple software programs have been 
developed to meet EASIER specifications, allowing 
schools to upload student files directly to the sys- 
tem. The state maintains approximately 28 servers, 
including eight different SQL servers. Distribution 
is carried out using Access programs that have 
been written to specifically allow agency staff to 
obtain, create, and distribute information. 

Auditing procedures depend on the nature of the 
data. Financial data are audited locally by each 
school district. Each school district is also audited 
externally by the state. Enrollment, for example, is 
audited down to the individual student’s residential 
status in order to establish school districts’ budgets 
and funding. Additional audits are run on teacher 
data, including their licensing, assignment, salary, 
and demographic information. Non-certified staff 
are reviewed in a similar manner. Each year about 
20 percent of the districts also receive site visits 
by consultants who review schools’ data profiles, 
look for potential problems, and discuss how the 
districts perceive their data and future goals. 

Staff throughout the Iowa Department of Educa- 
tion focus on data collection and maintenance. 
Staff in the curriculum and student areas carry 
out applied research and evaluation, as well as col- 
laborating with state and federal program employ- 
ees within their areas. Staff rely on a number of 
resources to determine data definitions, including 
handbooks and course classifications published by 
the National Center of Education Statistics (NCES) 
and materials issued by the National Forum of 
Education Statistics. 

Iowa has designated staff responsible for moni- 
toring data quality. The state also uses several 



methods to train all staff (and vendors) involved 
in data collection and reporting. Staff members 
receive ongoing statistical training in SAS, and 
the department has formed a SAS users group 
that also includes private industry. The state cre- 
ated the Iowa Communications Network (ICN), a 
two-way audio-video conference system. Training 
is conducted via the ICN in the summer and fall, 
and in addition a one-day data conference is held. 
NCES funding has supported five regional meet- 
ings, and Iowa personnel participate in the NCES 
Fellows Program. 

Defining state data needs 

How Iowa defines data needs 

The Division of Financial and Information Ser- 
vices is responsible for collecting student, finan- 
cial, and staff data and for defining data elements 
and collection methodologies. Personnel within 
the Division who focus on staff, curriculum, and 
students create the definitions for their areas of 
expertise. Federal requirements also dictate data 
definitions, and much of the state’s data collec- 
tion is now conducted to meet EDEN reporting 
requirements. 

Information and data priorities 

Iowa’s primary goal with respect to education 
data collection is to improve student achievement 
as measured by the Iowa Tests of Basic Skills 
(ITBS), the Iowa Tests of Educational Development 
(ITED), and other local assessments. The depart- 
ment also collects data to meet regulatory and 
accreditation responsibilities, including federal 
reporting requirements that must be met as part 
of the accountability component of No Child Left 
Behind (NCLB). The department strives to make 
education data available to those who need it, 
including schools and policy makers. As a result of 
NCLB, the data collected by the state is better able 
to inform policy decisions. 

Questions that the state’s educational data can 
address include: whether student achievement 
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is improving; which schools are achieving their 
goals; the characteristics of students; trends in 
enrollment; changes in program demand; where 
students go after graduation; and the distribu- 
tion of revenue and expenditures. Generally the 
department is able to respond to information 
requests, although queries regarding academic 
calendars and hours or days of instruction cannot 
be answered with currently available data. 

The remainder of this section reviews the kinds 
of questions the state of Iowa would most like its 
educational data to address in the short-term and 
looking further to the future, and highlights the 
information needs which senior state education 
agency officials currently consider to be particular 
priorities. Existing challenges which may affect 
the state’s ability to achieve these objectives mov- 
ing forward are discussed below. 

Establishing benchmarks. Iowa would like to 
gather more information on students’ transitions 
to postsecondary education and is in discussions 
with state universities and community colleges. 
While many of these institutions already report 
students’ information back to the local school dis- 
tricts, this is not a mandatory procedure. The de- 
partment is looking into developing a postsecond- 
ary data warehouse that will allow it to illuminate 
high school experiences and clarify in which high 
school content areas students are better prepared 
for college. Results could then be sent back to the 
high schools to help them improve their programs. 
A first committee meeting regarding this issue 
took place in November 2006, and a report was 
delivered to the legislature. 

Iowa recognizes the importance of understanding 
the characteristics of their students, including en- 
rollment and population changes. Some examples 
of this type of student information include data 
gathered on the Hispanic population and students 
who receive free or reduced price lunches. 

The biggest challenges for Iowa are not necessar- 
ily the need for more data, but the reporting and 
application of the data the state already collects. 



Currently, the most challenging data collections 
are the adequate yearly progress and the No Child 
Left Behind requirements. These data are difficult 
to clean, and the software used to determine a 
school’s position is exceptionally complex. Iowa 
also seeks a better longitudinal data system and is 
exploring options for building a data warehouse 
to help address these issues (see http://www.iowa. 
gov/educate/content/view/44/310/1/5/). 

Tracking progress. Iowa seeks better informa- 
tion regarding teacher professional development, 
although the state already maintains basic teacher 
demographic information and a database that re- 
cords what classes teachers are teaching and their 
licensure information. All of this data is gathered 
and cross-referenced with information obtained 
by school improvement consultants and the school 
districts. However, the state does not currently 
keep records regarding continuing education com- 
pleted by teachers. Similar records could also be 
kept on paraprofessionals working in the schools. 

Iowa is also interested in more information on 
whether students are completing a core curricu- 
lum, and further defining what the core curricu- 
lum encompasses. Obtaining this data is a current 
priority, and the information will be collected 
beginning in the 2007/08 school year. 

Defining who is a dropout remains problematic, 
although Iowa’s implementation of unique student 
identifiers for all students has greatly improved 
the accuracy of dropout rates. The department 
now compares students reported as dropouts 
by the school districts with student enrollment 
data from throughout the state. If a student has 
simply transferred to another school district, the 
department can alert the student’s original school 
district. In this way many school districts found 
they were reporting their dropout rates higher 
than they actually were. 

Like many other states, Iowa has had trouble cal- 
culating accurate graduation rates using estimates 
based on ninth grade attendance data, as these 
students can enroll, drop out, transfer, or take 
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more than four years to complete high school. Understanding state data challenges 



Iowa expects to report a more accurate graduation 
rate using data drawn from their unique student 
identifiers beginning in 2008. 

Documenting outcomes and their causes. The 
department is interested in producing maps of 
school districts and the areas individual schools 
serve on which variables such as achievement, tax 
revenues, or free and reduced-price lunches could 
be overlaid; such graphical representations could 
be particularly helpful to policy makers. Relevant 
geographic applications have been identified, but 
these cannot be employed with existing data, staff, 
and software resources. 

In addition to developing more data on postsec- 
ondary patterns, the department would like more 
information on the experiences of children before 
they enter kindergarten. Evidence shows that 
thriving preschools can have a tremendous effect 
on the overall school system. 

Enhancing capacity to use existing data resources. 
Funding for additional staff members would be the 
most helpful resource for enhancing Iowa’s edu- 
cational data system. The department also would 
be interested in purchasing a series of audits and 
business rules produced by the Center for Data 
Quality (C4DQ). These programs are run against 
existing files and indicate how clean the data are. 
Currently, the software licenses cost approximately 
$50,000 to $60,000, which the department is un- 
able to provide at this time. 

While the available products are sufficient for the 
state’s purposes at the moment, staff will need 
more training as new products are issued and 
more data are collected and reported. Some mem- 
bers of the department are already able to attend 
national and regional meetings; it would be desir- 
able to provide similar opportunities at the district 
and school levels. Iowa also would like to set up a 
data academy, such as the one created in Ohio, to 
help train local district staff in data quality issues. 
Participants would receive award certificates for 
completing a series of courses at the academy. 



The collection of consistent educational data has 
had considerable impact within the Iowa Depart- 
ment of Education. State education agency staff 
are able to provide legislators, school districts, and 
the general public with a picture of the educational 
situation in Iowa that is based on actual data. Such 
data have become an integral part of understand- 
ing the condition of education in the state and 
have had a significant impact on the state’s educa- 
tion policies. As a result there is growing demand 
for educational data to be easily accessible to both 
policy makers and parents, although concerns 
about student privacy are a serious issue. 

The department is able to collect needed data in 
a timely fashion, although resource constraints 
present some challenges in analyzing it. The 
number of staff in the department has been cut in 
half since the early 1990s even though the volume 
of work has increased. On the technology side, the 
state finds it difficult to train staff in new software 
products and to meet growing expectations about 
the accessibility of data on the Internet. A data 
warehouse and the implementation of SIF would 
help the department make the data more available. 

Generally, Iowa has not had problems complying 
with federal reporting requirements, though the 
requirements themselves are sometimes incon- 
sistent over time and/or with state requirements. 
The Office of English Language Acquisition, for ex- 
ample, has not always used consistent definitions, 
and the Office of Civil Rights requests information 
on special education children by their disability 
whereas Iowa classifies these children by the ser- 
vices they receive. Reporting difficulties may arise 
under EDEN as the state will need to solicit more 
detailed information than is currently collected. 
Iowa, along with other states, has requested fund- 
ing to create a staff position to help facilitate data 
collection for federal EDEN reports. 

Iowa encounters some problems in coding proce- 
dures, but fewer today than in the past. By insist- 
ing on standard coding statewide, the state has 
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Iowa currently bases its data definitions on NCES 
standards, but then develops its own data element 
dictionaries. The state is waiting to see how suc- 
cessful North Dakota is in its grant-funded project 
to allow states to modify codebooks and hand- 
books to fit the state’s definition. If the project is 
successful, local school districts might be able to 
make similar adjustments. Iowa also would like 
to move from data dictionaries to using metadata 
if funding becomes available. Secondary analysis 
of datasets should be possible with the available 
documentation; however, department staff and 
resources are fully occupied analyzing data that 
need to be reported. 



State data inventory 

Iowa's educational data system 

Information on Iowa’s educational data system, 
processes, and files can be obtained from the indi- 
viduals listed in table Dl. 

Key data elements 



eliminated many of the problems that previously 
occurred, although coding is still a concern (e.g., 
as the federal government requires more multiple 
reporting on race and ethnicity). Another area 
where coding problems occur is Iowa’s uniform 
course classification system. To alleviate these 
issues, Iowa would benefit from more financial 
support to enhance data quality training. Hir- 
ing a data steward or coordinator in every school 
district would be another means to reduce coding 
errors, and to help eliminate the problem of miss- 
ing data, which often are in fact incomplete data 
as districts find it increasingly difficult to submit 
information on time. 

The state has set data collection schedules, thus 
experiences no problems associated with incon- 
sistencies in the intervals between data collection. 
Data entry errors are not a serious problem either 
since the state provides data quality training 
to district staff so that they can use software to 
catch data entry errors. The department is able to 
respond to information requests promptly, and 
all staff are trained in SAS. The ability to provide 
information would be enhanced by the establish- 
ment of a data warehouse; this was described as an 
ideal mechanism for staff and the general public 
to have access to state data, and even be able to 
pull their own information and create their own 
tables. Such a warehouse also would make it easier 
to match records, although Iowa’s unique student 
identifier program and the EASIER database have 
made this fairly straightforward. 

Although inconsistencies in data definitions 
have been challenging for Iowa, the department 
has developed a single data dictionary to catch 
anomalies in definitions as the state consolidates 
data collections. Iowa also has alleviated definition 
inconsistencies by the implementation of a student 
record system whereby districts upload set data ele- 
ments with well-defined, consistent codes directly 
to the state. This system also alleviates the problem 
of inconsistencies in data definitions over time. 
However, the state does maintain some unique 
data sets, particularly the special education data set 
which is kept outside the Department of Education. 



The remainder of this section provides an over- 
view and highlights of student and teacher data 
currently collected and archived by the state of 
Iowa. For additional information see http://www. 
iowa.gov/educate/content/view/346/299/. 

Student data 

Demographics. Gender, date of birth, ethnicity/ 
race, and low-income status (identified as free and 
reduced lunch) are collected by BEDS at the student 
level. These data elements, however, are accessible to 
the public only at the district and/or building level 
at the Iowa Department of Education’s education 
statistics web site. For additional information see: 

• http://www.dataqualitycampaign.org/survey_re- 
sults/state.cfm?st=Iowa 

• http://www.iowa.gov/educate/component/ 
option,com_docman/task,cat_view/gid,129/ 
Itemid,55/ 
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TABLE D1 

Key contacts for more information on Iowa's data processes and files 



These data collection activities ... ... are currently the responsibility of: 



1 These data collection activities . . . 


... are currently the responsibility of: | 


Basic Educational Data Survey (BEDSy.Jhe BEDS is a survey tool 
used to collect staffing, student and policy data from all K-12 
public schools and accredited nonpublic K-12 schools. 


Bureau of Planning, Research, & Evaluation 

Betsy Lundy 

319-358-6206 

betsy.lundy@iowa.gov 

Marlene Dorenkamp 

515-281-5507 

marlene.dorenkamp@iowa.gov 


Project EASiER (Electronic Access System for iowa Education 
Records): Project EASIER is the initiative for ensuring effective 
methods for data collection. Specifically, Project EASIER 
involves transferring individual student records into a data 
repository, thereby resulting in timely data transfer and 
access for schools, postsecondary institutions and the Iowa 
Department of Education. 


Bureau of Planning, Research, & Evaluation Coleen 

McClanahan 

Project EASIER Manager 

515-281-7509 

coleen.mcclanahan@iowa.gov 


iowa Student Number (ISN)\ The ISN is used to identify every 
public school student from grades K though 12. The State ID 
System assigns the ISN, which is then used to submit data via 
Project EASIER. 


Bureau of Planning, Research, & Evaluation 

Carla Schimelfenig 

515-281-3111 

Carla.Schimelfenig@iowa.gov 


EDEN 

For additional information on the Education Data Exchange 
Network see http://www.ed.gov/about/offices/list/oig/ 
auditreports/al 1e0003.pdf) 


Office of the Director 
Judy Jeffrey, Director 
515-281-3436 
judy.jeffrey@iowa.gov 


State education agency submissions 


Office of the Director 
Judy Jeffrey, Director 
515-281-3436 
judy.jeffrey@iowa.gov 


• http://www.iowa.gov/educate/component/ 
option,com_docman/task,cat_view/gid, 131/ 
Itemid,55/ 


categories, except for disability status. For addi- 
tional information see: 


• http://www.iowa.gov/educate/content/ 
view/44/310/1/2/ 


• http://www.dataqualitycampaign.org/survey_re- 
sults/state.cfm?st=Iowa 


• http://ia-sb.org/humanresources/beds_%20pre- 
sentation.ppt 


• http://www.iowa.gov/educate/component/ 
option,com_docman/task,cat_view/gid,130/ 
Itemid,55/ 


Student special population status. The data for 
English proficiency status (ELL status), disability 
status (lEP indicator, Section 504 indicator, etc), 
economic disadvantage status (free lunch indica- 
tor and reduced-price lunch eligible indicator) and 
migrant status (migrant indicator) are collected at 
the student level by BEDS; however, the data acces- 
sible to the public on the education statistics web 
site is provided only at the district and/or building 
level for all the student special population status 


• http://www.iowa.gov/educate/component/ 
option,com_docman/task,cat_view/gid,393/ 
Itemid,55/ 

• http://www.iowa.gov/educate/component/ 
option,com_docman/task,cat_view/gid,342/ 
Itemid,87/ 

• http://www.iowa.gov/educate/content/ 
view/44/310/1/2/ 
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http://ia-sb.org/humanresources/beds_%20pre- 

sentation.ppt 

Enrollment. School, enrollment data, and mobility 
data (as identified as expulsion data) attendance 
data are collected by BEDS at the student level. The 
data are stored permanently by the state for use in 
subsequent years to determine continuous enroll- 
ment. These data elements, however, are accessible 
to the public only at the district and/or building 
level at the Iowa Department of Education’s educa- 
tion statistics web site. For additional information 
see: 

http://www.dataqualitycampaign.org/survey_re- 

sults/state.cfm?st=Iowa 

http://www.iowa.gov/educate/index. 

php?option=com_docman&task=cat_ 

view&gid=1298dtemid=55 

http://www.iowa.gov/educate/content/ 

view/44/310/1/2/ 

Course-taking. Iowa collects student-course 
taking and completion records at the student 
level using BEDS. Each course is designated a 
course number that is maintained throughout 
the system. Middle school and summer school 
classes taken for high school credit are not 
included in the course completion data. However, 
dual enrollment courses that meet high school 
requirements, but are taken from colleges and 
universities, are counted in the data collection. 
For additional information see: 

http://www.dataqualitycampaign.org/survey_re- 

sults/state.cfm?st=Iowa 

http://www.iowa.gov/educate/content/ 

view/44/310/1/1/ 

http://www.iowa.gov/educate/content/ 

view/44/310/1/2/ 

Special program participation. Early childhood 
program participation, individualized education 



program information, special assistance programs 
and honors programs/courses data are collected by 
BEDS at the student level. For additional informa- 
tion see: 

• http://www.dataqualitycampaign.org/survey_re- 
sults/state.cfm?st=Iowa 

• http://www.iowa.gov/educate/index. 
php?option=com_docman&task=cat_ 
view&gid=1188dtemid=55 

• http://www.iowa.gov/educate/component/ 
option,com_docman/task,cat_view/gid,517/ 
Itemid,55/ 

• http://www.iowa.gov/educate/component/ 
option,com_docman/task,cat_view/gid,342/ 
Itemid,87/ 

• http://www.iowa.gov/educate/content/ 
view/44/310/1/2/ 

Achievement. According to The Annual Condition 
of Education Report 2005 (p. 113), 4th, 8th and 
11th grade students take the Iowa Tests of Basic 
Skills (ITBS) and the Iowa Tests of Educational 
Development (ITED). Reading comprehension and 
math are assessed for grades three through eight 
and eleven, and science is currently assessed for 
8th and 11th graders. High school seniors take the 
ACT, SAT and AP exams. For additional informa- 
tion see: 

• http://www.dataqualitycampaign.org/survey_re- 
sults/state.cfm?st=Iowa 

• http://www.iowa.gov/educate/component/ 
option,com_docman/task,cat_view/gid,519/ 
Itemid,55/ 

• http://www.iowa.gov/educate/content/ 
view/44/310/1/2/ 

Graduation and dropout data. Using BEDS, Iowa 
collects student-level graduation data by diploma 
type. Dropout data are also collected and stored. 
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Furthermore, the state records where departing 
students go for grades 7-12, even if the students 
are not reenrolling or graduating. Iowa does not 
keep evidence to clarify if the students are drop- 
outs, transfers or missing students. There are not 
standards for the percent of departing students 
that a district should be able to locate. For addi- 
tional information see: 

• http://www.dataqualitycampaign.org/survey_re- 
sults/state.cfm?st=Iowa 

• http://www.iowa.gov/educate/component/ 
option,com_docman/task,cat_view/gid,518/ 
Itemid,55/ 

• http://www.iowa.gov/educate/component/ 
option,com_docman/task,cat_view/gid,516/ 
Itemid,55/ 

• http://www.iowa.gov/educate/content/ 
view/44/310/1/2/ 

Teacher data 

Demographics. Teacher demographic data, includ- 
ing teacher and administration salaries, experi- 
ence and age, may be collected from the area 
education agencies and districts once a year. For 
additional information see: 

• http://www.iowa.gov/educate/index. 
php?option=com_docman&task=cat_ 
view&gid=958dtemid=55 

Certification, qualifications, and professional 
development. The state of Iowa maintains data for 
teacher certification and qualifications but does 
not collect data on teacher professional develop- 
ment. For additional information see: 



• http://www.iowa.gov/educate/index. 
php?option=com_docman&task=cat_ 
view&gid=958dtemid=55 

Mobility and attrition. Neither teacher mobility 
nor teacher attrition data are collected by the state. 

Linking state data 

All students in Iowa are assigned a unique state- 
wide student number, which is collected by BEDS. 
These identifiers are used to link student-level 
records throughout the state’s databases. Likewise, 
Iowa teachers also receive a unique statewide 
teacher number. Iowa does not match records 
across teachers and students. (For additional 
information see http://www.dataqualitycampaign. 
org/survey_results/state.cfm?st=Iowa). 

Data quality assurance 

Iowa relies on separately collected and verified 
databases for the student demographic informa- 
tion. According to the Data Quality Campaign, the 
state does have procedures to prevent two different 
individuals from receiving the same ID. Iowa also 
has procedures to prevent the same student from 
obtaining/receiving a different ID when she/he 
changes districts. Likewise, the state has proce- 
dures established to prevent teachers from being 
assigned two IDs. The state audits and performs 
statistical checks on data submitted by the school 
districts. Currently, the state does not have an 
established system to investigate the accuracy of 
data flagged by the statistical checks. However, they 
have established certain criteria to help determine 
when data submitted by the school districts are 
prone to be in error. Iowa does not impose conse- 
quences on school districts that do a poor job gath- 
ering or submitting data, nor on school districts 
that do not accurately account for missing students. 
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APPENDIX E 
MICHIGAN 



Information needs assessment 

Data collection and quality assurance procedures 

In Michigan the collection, management, and re- 
porting of educational data from K-12 entities are 
coordinated by the Center for Educational Perfor- 
mance and Information (CEPI). Responsibilities of 
this Center include to: “coordinate the collection of 
all data required by state and federal law ... in the 
most efficient manner possible in order to reduce 
the administrative burden on reporting entities”; 
“establish procedures to ensure the validity and 
reliability of the data and the collection process”; 
“develop state and model local data collection poli- 
cies, including . . . policies that ensure the privacy 
of individual student data”; “provide data in a use- 
ful manner to allow state and local policymakers 
to make informed policy decisions”; and “provide 
reports to the citizens of this state to allow them 
to assess allocation of resources and the return on 
their investment in the education system of this 
state” (see Section 388.1694a of the State School 
Aid Act of 1979^). 

CEPI has identified as its mission “to become the 
single source for the most comprehensive, accurate 
and useful information about the performance 
of Michigan’s public schools and students” (see 
http://www.michigan.gov/cepi/0,1607,7-113-985- 
3493-,00.html). The Center works with the De- 
partment of Information Technology to determine 
efficient and effective mechanisms for complying 
with state and federal data requirements, includ- 
ing using tools provided by the National Center 
for Education Statistics (NCES) to enhance data 
collection. CEPI manages the Michigan Education 
Information System (MEIS), a “data warehouse 
system used by school districts to submit data 
to the state and by CEPI to combine, store, and 
report that data” (see http://www.michigan.gov/ 
cepi/0,1607,7-113-986— ,OO.html). The MEIS is 
comprised of Microsoft-based software providing 
tools for collecting, maintaining, and reporting 



data. Within the MEIS, the School Code Master 
(SCD) contains the directory of school information 
used to link the MEIS datasets: a Single Record 
Student Database (SRSD), including “discrete 
information about individual students such as age, 
gender, race and ethnicity, and program participa- 
tion”; a Registry of Educational Personnel (REP), 
that collects “basic employment elements relat- 
ing to school personnel, such as certification and 
degrees held, school and grade/subject assignment, 
length of service, and salary”; a School Infrastruc- 
ture Database (SID), including “information about 
safety practices and incidences of crime in public 
schools. Title I Schoolwide Programs and Dual En- 
rollment”; a Financial Information Database (FID), 
including “information from districts’ annual 
financial reports, balance sheets, revenues, district 
expenditures and school expenditures”; and the 
Administrator Data Review (ADR) application, 
developed by CEPI to provide “useful information 
back to the intermediate school district (ISD), local 
education agency (LEA), district and public school 
academy (PSA) district staff members from the 
educational data they submit to CEPI each year.”^ 

CEPI ensures that data collection activities meet 
federal and state data requirements and abide by 
the requirements of the Headlee Amendment, 
which places restrictions on newly mandated 
activities or services required by the state. So, for 
example, while the School Infrastructure Data- 
base (SID) “has the capability to include infor- 
mation about technology (hardware, networks, 
connectivity, distance learning etc.) and school 
structure (physical construction, capacity and use 
elements)” these data are not currently collected, 
as presently they “are not required by the state or 
federal government.”^ 

CEPI appoints an individual to oversee data col- 
lection, work with the Department of Information 
Technology to collect and maintain data using 
the MEIS, and provide customer support to assist 
districts in managing their data. Data quality is 
ensured through a number of audits at the district 
and the state level. Intermediate districts conduct 
audits to make sure their data records do not 
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count individual students more than once. Dis- 
tricts also sample schools to conduct field audits 
and desk audits; their purpose is to determine 
whether student enrollment data are accurate and 
to confirm whether the state provides aid to that 
school district. CEPI provides data to the Office of 
Audits in the Department of Education to audit for 
students with full-time status; the Office of Audits 
provides technical assistance in auditing pupil en- 
rollment. At the state level, audits are conducted to 
ensure the validity and reliability of data submit- 
ted by the districts. Error checks are run based on 
established parameters; consistency with historical 
data is established, and databases are triangulated 
to test for accuracy and reliability. (The Michigan 
State Department of Education does not, however, 
have the statutory authority to audit districts for 
data quality before data are submitted to the state.) 

Application developers in the Department of 
Information Technology (DIT) create the software 
needed to make Michigan’s education data avail- 
able and accessible. DIT also has quality assur- 
ance staff to develop test plans and test cases for 
software developers to write and use to check the 
quality of the data. 

The CEPI data administrator manages data, adds 
and derives variables, develops crosswalks and 
combines datasets as necessary to ensure data are 
compliant with federal requirements. Staff of the 
Michigan Department of Education ensure data 
comply with federal requirements before they are 
submitted to the U.S. Department of Education. 
State education agency officials avail themselves of 
training sessions provided by the National Center 
for Education Statistics and other federal agencies 
(e.g., on the Education Data Exchange Network, 
EDEN, and Common Core Data, CCD), and attend 
NCES National Forum on Education Statistics and 
Management Information System conferences to 
keep informed on best practices of data use and 
management. State education agency officials also 
receive updates from the Education Information 
Management Advisory Consortium (IMAC), affili- 
ated with the Council of Chief State School Officers 
(CCSSO). 



Key reports and documentation on the data col- 
lection process developed by CEPI are available 
on: the CEPI web site; the Michigan Association 
of School Boards web site; and the School Matters 
web site, a service of Standard & Poor’s (see www. 
schoolmatters.org). CEPI maintains its own web 
site to provide data on how schools and districts 
are performing (see http://www.michigan.gov/ 
cepi/0,1607,7-113-985-3496-,00.html). The Michi- 
gan Association of School Boards (http://www. 
masb.org/) also provides assistance to local school 
boards to assess their data and compile school 
profiles. 

Defining state data needs 

How Michigan defines data needs 

Working with CEPI to define Michigan’s data 
needs are three distinct groups of individuals. The 
first group includes representatives of eight state 
agencies that participate in data governance, with 
a data policy committee comprised of decision- 
makers from those eight agencies. Also contrib- 
uting to the definition of the state’s educational 
data needs are the members of the Data Managers 
Working Group (DMWG); these individuals work 
directly with federal program offices and state of- 
fices to respond to state and federal mandates and 
articulate compliance needs. A third group is com- 
posed of other stakeholders who provide valuable 
input to the needs definition process, including 
K-12 entities, educators, superintendents, interme- 
diate school district superintendents, individuals 
responsible for data entry, and members of the 
public, advisory groups, and focus groups. 

Procedurally DMWG members identify and 
advise the data policy committee of data collection 
measures needed to comply with state and federal 
mandates. A key function of the data managers 
working group is to identify potential duplicate 
data collections. DMWG members advise the Data 
Policy Committee on collection measures neces- 
sary to comply with state and federal mandates. 
The data policy committee works with the data 
managers to define procedures to enhance the 
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efficiency and effectiveness of data collection and 
use. Key stakeholders provide feedback on current 
processes and input on what, from their perspec- 
tives, are the most useful formats for accessing 
data — advice which is described as critical in cur- 
rent efforts to build a state data warehouse capable 
of longitudinal reporting of all the data Michigan 
collects. 

Information and data priorities 

The vision of Michigan’s data governance pro- 
cess is to “collect once, store once, and use many 
times.” Importantly, ‘use many times’ is recog- 
nized to mean ‘share many times’; thus a key 
consideration is how data can be shared to reduce 
the reporting burden for districts. Accordingly key 
priorities are to establish one system to collect and 
warehouse data to provide information in a user- 
friendly, accessible format. 

Student data are collected three times a year, per- 
sonnel data are collected twice, and financial data 
are collected once in the fall after the school year 
has been completed. These data are used to look 
at student performance at the district, school, and 
student levels; e.g., the Michigan Department of 
Education provides resources to generate scatter- 
plots at the district level and maintains historical 
student data in a secure web site. School report 
cards provide assessment results as an indicator of 
school performance; these report cards are readily 
available to the public on the Michigan Department 
of Education web site (https://oeaa.state.mi.us/ayp/ 
index.asp). Longitudinal school enrollment, atten- 
dance, demographic and outcome data are avail- 
able at the student level. Currently, however, data 
are not collected that would provide information 
for assessing instructional intervention at the class- 
room level. It was noted such data would be helpful 
to the state for identifying factors (e.g., resources, 
curriculum, teaching experience) that encourage 
students to stay in school and are associated with 
positive student learning outcomes. 

The remainder of this section reviews the kinds of 
questions the state of Michigan would most like its 



educational data to address both in the short-term 
and looking further to the future, and highlights 
the information needs which senior state educa- 
tion agency officials currently consider to be par- 
ticular priorities. Existing challenges which may 
affect the state’s ability to achieve these objectives 
moving forward are discussed below. 

Establishing benchmarks. One of the challenges 
in using state data to compare Michigan’s edu- 
cational progress with that of other states in the 
nation is the variation in data definitions currently 
employed among the states. For example, one 
state may define truancy as 10 or more unexcused 
absences, while another may define it as 20 or 
more unexcused absences. It is difficult to com- 
pare graduation outcomes and dropout challenges 
across states given different formulas for deter- 
mining graduation and dropout rates. Assess- 
ment scores also cannot be readily compared, as 
different measures and formulated scores are used 
among states. 

Determining how various subgroups of students 
are performing in school is a priority for the state 
of Michigan. State education agency officials have 
a perspective on how to effectively measure a 
student’s fullest academic potential: to establish 
benchmarks and obtain a complete profile of 
students’ academic performance and capabili- 
ties, individual student data must be collected 
and measured in the context of the environment 
and resources available to each student. Subgroup 
identification provides only a partial profile of 
a student; e.g., simply identifying students on 
the basis of their racial background provides an 
incomplete understanding of the individual-level 
characteristics that may affect their academic 
experiences. State education agency officials 
note it is critical to examine all factors affecting 
students’ learning experiences, including, ideally, 
data on environmental and available resources 
not obtained via current data collections. More 
detailed student profiles could be extremely infor- 
mative in determining the most effective academic 
programs and practices to offer a diverse group 
of students. A legislative mandate to collect such 
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data is crucial to the ability to develop these more 
comprehensive student profiles. 

Tracking progress. Michigan’s most pressing 
information needs include gathering more data 
on teacher training, teacher professional develop- 
ment, curriculum development, and transitions to 
postsecondary education. Information on teacher 
preparation programs and where teachers teach 
after completing teacher training is collected, but 
additional data on individuals’ teaching experi- 
ences and how these may influence retention or 
attrition from the profession are desirable. Such 
information is crucial for better understanding 
the tools and training that best prepare teachers 
for their classrooms. Additional data on profes- 
sional development and the ability to link such 
data to student outcomes would also be beneficial. 
(Currently the state gathers data on the prepara- 
tion programs individual teachers attended, and 
where they were licensed; these data are provided 
to the Office of Professional Preparation Services 
with identifying names, but without their social 
security number or any identification number to 
protect anonymity. Without any identification 
number, there is no structured system to provide 
this data to the districts.) There is also an interest 
in exploring any relationships between teacher 
retention and student outcomes. 

Curriculum development is another priority topic 
in Michigan. The state provides standards and 
benchmarks for its curricular programs, and 
these standards are helpful for identifying what 
assessments focus on when evaluating student 
performance. A standardized method for identify- 
ing lesson plans and materials instructors use, and 
consistent course naming across schools would 
make it possible to assess how certain curricular 
programs fare in various school settings, and to 
identify which curriculum programs are effective 
with which students. 

Interest was also expressed in more and better 
data to assess transitions to postsecondary educa- 
tion. A state education agency official described 
two obstacles to collecting such data. First, 



higher education institutions do not use a unique 
identification code for student data. This makes it 
challenging to link K-12 education student data to 
postsecondary education data. Second, education 
institutions and education policymakers need to 
collaborate in working within the constraints of 
the Family Educational Rights and Privacy Act to 
allow K-12 education and higher education institu- 
tions to share data while still protecting the rights 
of students and their families. 

No additional enhancements were suggested to the 
information collection processes Michigan plans 
to provide data for calculation of graduation or 
drop-out rates. A new method from the National 
Governor’s Association Compact on Graduation 
Counts has been used to track senior high school 
students since 2002. This method is expected to 
provide not only more data on graduation and 
drop-out rates, but also to enhance capacity to 
monitor data entry at the district level. Data qual- 
ity is anticipated to improve significantly, as raw 
data rather than information based on estimation 
formulas will be collected. NCES definitions will 
also be used for drop-outs. 

Enhancing capacity to use existing data resources. 
As noted above, at the state level, Michigan utilizes 
a variety of resources to check the validity and 
ensure the accuracy of state databases. Further 
enhancements to quality control procedures are 
limited by the inability to establish the accuracy 
of school district data before it is submitted to the 
state database. Critical here is the absence of a 
statutory authority to audit the quality of school 
data. Should such an authority be established, ad- 
ditional resources would also be required to enable 
state staff to assist districts in auditing the quality 
of their data. State staff amount to less than twenty 
people, and there are 820 school districts to audit. 
Additional tools to check data entered at the 
school district level would also be desirable. 

Understanding state data challenges 

The importance of accountability has shifted from 
using aggregated data collected at a specific point 
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in time to using disaggregated data collected over 
a period of time. Districts are making more con- 
certed efforts to collect valid and reliable disaggre- 
gated data by using more systematic procedures. 
This results in districts becoming more skilled in 
their resources to collect and maintain data on 
student outcomes and teaching factors. These data 
can be instrumental in facilitating discussions and 
making decisions on how to better address the 
educational needs in the state. 

Although there are working systems in place to 
collect disaggregated data, the state would benefit 
greatly from and has initiated plans to enhance its 
data warehouse. Building a longitudinal system to 
house data and make it available in an accessible 
and useful format to enhance policy and practice 
decisions is a high priority. Limited staff, exper- 
tise, time, and funding are challenges in develop- 
ing this warehouse. Once all the necessary data are 
collected and stored, priorities include determin- 
ing what types of reports would be most useful to 
the districts and other stakeholders (school board 
members, educators, policymakers, teachers, 
parents). 

Resolving remaining inconsistencies (e.g., in data 
definitions across datasets and over time) and 
developing the necessary information process- 
ing storage space are other priorities in Michigan. 
Inconsistencies in data definitions can be challeng- 
ing (e.g., the lack of consistent definitions of race/ 
ethnicity categories for multi-racial individuals). 
Data definitions can change when funding sources 
shift, as different funders may employ different 
definitions. One of the possible ways to address 
this obstacle would be for funding establishments 
to begin developing their programs first with data 
collection, and then develop the program based on 
the feasibility of the data collection. Efforts on the 
part of the federal government to be consistent in 
requesting data from the states are appreciated, as 
this allows the state to be consistent in requesting 
data from the districts. Michigan is able to comply 
with federal reporting requirements, although pro- 
gram offices continue to work to improve their stu- 
dent data collection activities and data reporting. 



The state is also concerned to address inconsis- 
tent coding procedures that may occur across 
items, datasets, and/or over time. Such challenges 
are particularly likely to arise when data are 
not consistently defined and when reports have 
short deadlines, so that data documentation may 
not be completed until after reports are submit- 
ted. Another challenge is inconsistencies in data 
definitions across states; standardization of data 
definitions across states would help to ensure the 
consistency of coding procedures across items, 
datasets, and/or over time. 



State data inventory 

Michigan's educational data system 

Several key agencies develop and maintain the 
structure of Michigan’s educational data sys- 
tems: the Michigan Department of Education, 
the Center for Educational Performance and In- 
formation of the Office of the State Budget, the 
Department of Labor and Economic Growth, 
the Office of the Governor, and the Department 
of Information Technology. These agencies col- 
laborate to support the collected data systems 
known as the Michigan Education Information 
System. 

Six databases are maintained in the Michigan edu- 
cational data warehouse, the Michigan Education 
Information System: 

• The Single Record Student Database (SRSD) is 
a collection of each student’s age, gender, race/ 
ethnicity, and program participation. SRSD data 
are collected three times a year, and is used to 
meet the No Child Left Behind Act of 2001 and the 
adequate yearly progress requirements. 

• The Registry of Educational Personnel (REP) col- 
lects information on school personnel, including 
the Michigan Department of Education’s teacher 
certification audit. School personnel data and 
reports are collected from school districts twice a 
year. 
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• The School Infrastructure Database (SID) is a 
collection of data on Title I Schoolwide Programs 
and Dual Enrollment, safety policies and practices, 
school prevention programs, disciplinary prob- 
lems, and safety/disciplinary incidents. 

• The Financial Information Database (FID) collects 
data on districts’ annual financial reports, balance 
sheets, revenues, and district and school expendi- 
tures. Data are collected in compliance with the 
Michigan Public School Accounting Manual Chart 
of Accounts. 

• The School Code Master (SCM) is the state school 
directory. It includes general directory informa- 
tion and official district and school identifica- 
tion codes for all K-12 public schools. SCM data 
complies with the NCFB and the Michigan 
Department of Education’s accreditation initiative. 
Education YES!. 

• The Administrator Data Review (ADR) is a data- 
base of CEPI data given access to the intermediate 
school districts, local education agencies, local 
education agencies, and public school academies. 



Key data elements 

Michigan continues to make great strides in 
collecting key data elements. Their databases 
include student-level enrollment, demographic, 
program participation data, and student-level 
graduation and dropout data. As reported by the 
Data Quality Campaign, Michigan is capable of 
using a unique statewide student identification 
system to link student data across key databases 
over multiple years, as well as match individual 
students’ test data across years to assess aca- 
demic growth. 

The remainder of this section provides an over- 
view of student and teacher data currently col- 
lected and archived by the state of Michigan. Key 
contacts for additional information on Michigan’s 
data collection systems are listed in table El. For 
additional information see the CEPI direct web 
link (http://www.michigan.gov/cepi/0,1607,7-l 13 
-985--,00.html) and the MEIS direct web link 
(http://www.michigan.gov/cepi/0,1607,7-113-986- 
-,00.html). 



TABLE El 

Key contacts for more information on Michigan's data collection activities 



These data collection activities . . . 



... are currently the responsibility of: 



• Single Record Student Database (SRSD) 

• Registry of Educational Personnel (REP) 

• School Infrastructure Database (SID) 

• Financial Information Database (FID) 

• School Code Master (SCM) 

• Administrator Data Review (ADR) 



Center for Educational Performance and Information (CEPI) 
Hannah Building, 2nd Floor 
608 West Allegan Street 
Lansing, Michigan 48913 
Phone: (517) 241-2374 
Fax: (517) 335-0488 
Email: CEPI@michigan.gov 
Web site: www.michigan.gov/cepi/ 

The Department of Information Technology (DIT) Client 
Service Center is available to address questions about CEPI 
applications. 



DIT Client Service Center 

235 South Grand, Suite 304 

Lansing, Michigan 48913 

Phone: (517) 335-0505 

Fax: (517)241-8439 

Email: Help-Desk@michigan.gov 

Web site: http://michigan.gov/dit/0,1607,7- 139-30629— 

,OO.html 
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Student data 

Demographics. CEPI collects key demographic data 
elements three times a year (see http://www.michi- 
gan.gOv/documents/CycFldSub0506_157950_7. 
pdf for the Cycles of Field Submission documen- 
tation). Gender, date of birth, race/ethnicity are 
collected three times a year, and low-income status 
is collected twice a year. CEPI cleanses, structures, 
and stores these data into MEIS collections. In 
addition to CEPI and MEIS, the MDE and CEPI 
submit aggregated data to the National Center for 
Education Statistics (NCES) as well as Standard 
& Poor’s SchoolMatters.com web site (see http:// 
WWW. schoolmatters . com/) . 

Student special population status. CEPI collects 
various student special population status data 
elements, including Limited English Proficiency 
status, special education program participa- 
tion, supplemental nutrition eligibility status, 
migrant status and participation in other federal 
and state programs through the SRSD data col- 
lection. Most of these data elements are collected 
in the fall, spring, and at the end of the school 
year, and are maintained by CEPI. CEPI, the 
MDE, NCES, and SchoolMatters.com provide 
access to reports and data on special program 
participation. 

Enrollment. Michigan’s enrollment database 
includes students’ gender, ethnicity, low-income 
status, English language learner status, operating 
district, resident district and attendance. All these 
data elements are collected three times a year by 
CEPI and are maintained in the SRSD data collec- 
tions. The enrollment data are permanently stored 
by the state so it is possible to monitor continuous 
enrollment over multiple years and these down- 
loadable data sets are available from the CEPI and 
NCES web sites. 

Course-taking. A limited number of course codes 
exist for the SRSD and REP data sets, but a consis- 
tent statewide course numbering system has not 
yet been developed, and course completion data 
(e.g., summer school courses completed for high 



school credit) are not collected and maintained at 
the state level. 

Special program participation. CEPI collects 
data on early childhood program participation, 
individualized education program information, 
special assistance programs, and Honors program/ 
courses (Advanced Placement courses included). 
Data are collected in the fall, spring and the end of 
the school year, and these data are available from 
the CEPI and NCES web sites. 

Achievement. The Michigan Educational Assess- 
ment Program (MEAP) is the state-wide assess- 
ment that evaluates third- through ninth-grade 
student performance in English, Language Arts, 
Mathematics, Science, and Social Studies. The 
MDE Office of Educational Assessment & Account- 
ability (OEAA) maintains student-level data and 
works with CEPI to identify and track students. 

The state also collects and maintains student-level 
data on each untested student in a tested grade, but 
does not collect explanations for why the student 
was not tested. These data are permanently stored 
for evaluating student achievement and academic 
progress over multiple years. Student-level test 
records also include demographic data, such as 
gender, race, ethnicity, low-income status, and 
English proficiency status. CEPI, the MDE, and 
SchoolMatters.com provide data access on student 
achievement. Student-level college readiness test 
scores (e.g., AP, SAT, and ACT exam scores) are not 
collected or maintained by the state. 

Graduation and dropout data. CEPI collects 
student-level graduation data, including diploma 
type, in the fall, spring, and at the end of the year. 
Also collected are student-level dropout data (in- 
cluding information on where un-enrolled seventh 
though twelfth grade students who do not gradu- 
ate go) and information on students who receive 
their GED or who are able to graduate after drop- 
ping out or transferring schools. All these data 
are stored in the SRSD data collections and are 
maintained and made available by CEPI. Gradua- 
tion and dropout data are also submitted to NCES 
and SchoolMatters.com for publication. 
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Teacher data 

Demographics. The REP collects personnel data, 
including demographic; certification, qualifica- 
tion and professional development; and mobility 
and attrition variables. These data are collected 
twice each year; CEPI, NCES, and SchoolMatters. 
com provide access to these data. Data variables 
include (see http://www.michigan.gov/documents/ 
RecLay0606_161977_7.pdf): 

• teacher’s name 

• social security number 

• credential license number 

• date of hire 

• school assignment (i.e., at which school the teacher 
works) 

• teaching position assignment 

• grade assignment 

• full-time equivalency status 

• wage 

• function code 

• highly qualified 

• academic major 

• academic minor 

• administrator continuing education 

• number of classes taught 

• Title I and Title II, Part A teacher status 

• funded position status 

• date of birth 



• gender 

• racial/ethnicity 

• highest educational level 

• type of credential 

• date credential issued 

• date of expiration of credential 

• hours of professional development 

• employment status 

• date of termination of employment 

• personnel identification code 

• full-time base annual salary 

• non-Michigan sponsoring institution 
Linking state data 

Multiple years of student-level demographic, 
enrollment, and achievement data can be linked. 
To date, no system has been implemented to link 
teacher and student data, and there is currently 
no common student identifier available to match 
student-level K-12 records to student-level higher 
education enrollment records. 

Data quality assurance 

DIT maintains the infrastructure to manage and 
store data for access by interested stakeholders. 
CEPI and MDE use network technology to collect, 
transfer, and report data. Agencies participating 
in educational data governance collaboratively 
develop standards to protect confidentiality of 
student-level data, standards for the collection 
of data, common definitions, and procedures to 
facilitate data-sharing among agencies. The state 
has procedures to ensure that neither teachers nor 
students have multiple identification numbers. As 
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Notes 



of 2006/07, the state has an audit system to review 
the accuracy of submitted FTE data in the aggre- 
gate, although as noted above there is currently no 
system to conduct on-site quality checks. 



1. Retrieved online 6 Jan 2007 from 
http://www.legislature.mi.gov/ 
(S(a0jl31fy2sp2vl55z2lwlz45))/mileg.asp 
x?page=getobject&objectname=mcl-388- 
1694a8i:queryid=1613767&highlight. Michi- 
gan’s Center for Educational Performance 
and Information was created within the office 
of the Michigan state budget director in the 
department of management and budget. For 
additional information on CEPI see Public Act 
191 of 2001 and the Center’s web site at www. 
michigan.gov/cepi. 



2. See the “MEIS at a Glance” (at http://www.mich- 
igan.gov/documents/Flow_Chart_34792_7. 
xls); “SSRD Overview” (at http://www.mi.gov/ 
cepi/0,1607,7-113-986_10481— ,00.html); 
“Registry of Educational Personnel (REP): 

New to the REP?” page 2 (at http://www. 
mi.gov/documents/NewToREP1004_107313_7. 
pdf); “SID Overview” (at http://www.mi.gov/ 
cepi/0,1607,7-113-986_10482— ,00.html);“FID 
Overview” (at http://www.mi.gov/cepi/0,1607,7- 
113-986_10484— ,00.html); and “ADR Over- 
view” (at http://www.mi.gov/cepi/0,1607,7-113- 
986_38364— ,00.html). 

3. See “SID Overview” (at http://www.mi.gov/ 
cepi/0,1607,7-113-986_10482— ,OO.html). 
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APPENDIX F affect the state’s ability to achieve these objectives 

MINN ESOTA moving forward are discussed below. 



Information needs assessment 

Data collection and quality assurance procedures 

Minnesota is in the process of creating an educa- 
tional data warehouse and expects to have 80 per- 
cent of the state’s major data systems warehoused 
by the end of June 2007. Staff are being trained 
to organize data and access it more centrally, 
although providing secure and variable access 
to stored data is technically complex and may 
take another six months to complete. Validation 
checks are built into all data collection efforts, and 
the state is working to make sure that the same 
piece of data is collected one time only to improve 
reliability 

Defining state data needs 

How Minnesota defines data needs 

Most data collected by the Minnesota Department 
of Education (MDE) are based on either state or 
federal reporting requirements. The Research 
and Assessment Division then works with other 
divisions to define data elements to be collected, 
usually by taking a federal data definition and 
refining it within the state context. 

Information and data priorities 

Minnesota’s primary goals with respect to its 
educational data are to centralize the data, ensure 
the data are catalogued, and make the data more 
accessible to a variety of end users (ranging from 
districts to researchers). 

The remainder of this section reviews the kinds of 
questions the state of Minnesota would most like 
its educational data to address in the short-term 
and looking further to the future, and highlights 
the information needs which senior state educa- 
tion agency officials currently consider to be par- 
ticular priorities. Existing challenges which may 



Establishing benchmarks. Minnesota would like to 
be able to compare data across states but remains 
wary of contextual differences which limit the 
comparability of data even when definitions are 
more or less the same. Exit codes, for example, can 
be applied, collected, and edited in different ways 
by different states. Comparing various sub-groups 
or sub-populations within the state also is a goal 
of the MDE. The state provides districts with 
uniform codes, trains administrators on reporting 
standards, and has appropriate editing options 
and quality controls. A remaining challenge is to 
create sensitivity throughout the MDE to state and 
federal differences in data definitions. 

Tracking progress. There is growing interest 
statewide in obtaining more or better information 
about the effectiveness of specific programs, teach- 
ing practices, and curricular reform. However, 
establishing consistent definitions and ensuring 
consistent data collection remain problematic. 

Documenting outcomes and their causes. Min- 
nesota is able to collect quality data on particular 
outcomes of interest such as academic achieve- 
ment, dropout rates, and transitions to postsec- 
ondary education. However, a recurring challenge 
is coordinating state efforts to collect such data 
with potentially conflicting requirements of 
federal agencies. The federal offices for Special 
Education (the U.S. Department of Education’s 
Office of Special Education and Rehabilitative 
Services) and the Title I program (the U.S. Depart- 
ment’s program office for Student Achievement 
and School Accountability Programs) both require 
states to collect assessment data, for example, but 
have different inclusion rules. Other agencies or 
programs (such as EDEN) may collect the same 
data but require different collection schedules. 

Enhancing capacity to use existing data resources. 
The MDE obtained a $3 million grant over three 
years to build a longitudinal data warehouse. 

These funds are expected to cover only a fraction 
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of the real cost of the system. In addition to funds 
for technical restructuring, Minnesota would wel- 
come funds for staff to coordinate federal report- 
ing requirements at the state level. 

Understanding state data challenges 

The MDE has identified two major issues that affect 
its ability to provide high-quality, longitudinal 
data. The first is building the data warehouse and 
providing secure access to different users. Related 
challenges include inconsistencies in coding proce- 
dures across items and over time (a data dictionary 
is being constructed); inconsistent intervals in data 
collection (the data warehouse will eliminate most 
of these); inconsistencies in the treatment of miss- 
ing data; inability to match records or link existing 
data sets (the data warehouse will resolve this); 
and data entry errors. More federal funding for 
building technical capacity or infrastructure would 
ensure Minnesota’s ability to address these issues. 

The second major issue for the MDE is a concern 
with the consistency of federal reporting require- 
ments. State education agency officials have noted 
that there have been major improvements over 
the last year or two, but would welcome addi- 
tional federal efforts to ensure the coherence and 
consistency of requests for state data from federal 
agencies. 

State data inventory 

Minnesota's educational data system 

The Minnesota Department of Education Data 
Center has been created to provide a repository 
of information about Minnesota schools. Several 



years of data have been compiled describing gen- 
eral characteristics of each school district. In most 
cases, data files for the current year are available 
and some extend back to the 1989/90 school year. 
These files are provided in several formats for users 
who may wish to load the data into their own soft- 
ware for analysis or reporting purposes. Reports 
and data files are made available and posted online 
after the data are finalized. Normally, the depart- 
ment finalizes student and staff data in March or 
April of each year. Periodically, data files are up- 
dated with minor revisions to some districts. More 
information about Minnesota’s educational data 
collections can be obtained through the contacts 
in table El. (For additional information see http:// 
cfl.state.mn.us/datactr/index.htm; http://www.ohe. 
state.mn.us/index.cfm.) 

Key data elements 

Following is an overview of student and teacher 
data currently collected and archived by the state 
of Minnesota. 

Student data 

Demographics. Minnesota collects student demo- 
graphics annually, including gender, age, ethnicity, 
race and English proficiency. The state does not 
collect date of birth, low-income status, disabil- 
ity status, or migrant status data. For additional 
information see: 

• http://cfl.state.mn.us/datactr/enroll/index.htm 

• http://www.ohe.state.mn.us/tPg.cfm 
?pageID=760&1534-D83A_1933715A= 
c7d2d499261b4237 



TABLE FI 

Key contacts for more information on Minnesota's data processes and files 



1 These data collection activities . . . 


... are currently the responsibility of: I 


Data submissions and downloads 


Cathy Wagner, 651-582-8688 or 
mde.data-downloads@state.mn.us 


Staff automated reporting (STAR) 


Dan Bittman, 651-582-8807 or mde.star@state.mn.us 
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http://cfl.state.mn.us/datactr/enroll/index.htm 

http://cfl.state.mn.us/datactr/language/index.htm 

Enrollment. Each year, Minnesota gathers student- 
level enrollment data, which is permanently stored 
to be subsequently used to determine continuous 
enrollment, student mobility, and attendance. For 
additional information see: 

http://cfl.state.mn.us/datactr/enroll/index. 

htm#school 

http://cfl.state.mn.us/datactr/enroll/index.htm 

http://cfl.state.mn.us/datactr/mobility/index.htm 

http://cfl.state.mn.us/datactr/attend/index.htm 

Course-taking. Minnesota does not collect data 
regarding student course taking. 

Special program participation. Minnesota does 
not collect data regarding student special program 
participation. 

Achievement. The state of Minnesota collects and 
stores data on student-level test results. These data 
are permanently stored to help determine aca- 
demic progress. In addition, Minnesota maintains 
individual records on all untested students in 
tested grades. SAT, ACT, and AP exam results are 
also collected and permanently stored by the state. 
For additional information see: 

http://www.dataqualitycampaign.org/survey_re- 

sults/state.cfm?st=Minnesota 



• http://www.ohe.state.mn.us/ 
mPg.cfm?pagelD=798&1534- 
D83A_1933715A=3d7acb42609b6ffb 

Graduation and dropout data. Minnesota annu- 
ally assembles student-level graduation data by 
diploma type. Likewise, the state collects student- 
level dropout data each year. The state does not 
track information on departing students. For 
additional information see: 

• http://cfl.state.mn.us/datactr/grads/index.htm 

• http://www.ohe.state.mn.us/tPg.cfm 
?pagelD=14648d534-D83A_1933715A= 
79dfda9979826da 

• http://cfl.state.mn.us/datactr/drops/index.htm 
Teacher data 

Demographics. Minnesota annually collects data 
on teacher gender and ethnicity by salary, assign- 
ment, and category. Age data also are collected. 

For additional information see: 

• http://education.state.mn.us/mde/Data/Data_ 
Downloads/StafT_Data/Licensed_StafT/index.html 

• http://cfl.state.mn.us/datactr/staff/index.htm 

Certification, qualifications, and professional 
development. Minnesota annually collects data on 
teachers’ years of experience and highest degree 
obtained. For additional information see: 

• http://cfl.state.mn.us/datactr/staff/index.htm 

Mobility and attrition. Minnesota does not collect 
data regarding teacher mobility or attrition. 



http://www.ohe.state.mn.us/mPg.cfm 

?pagelD=793&1534-D83A_1933715A= 

e39ede2d5c6064cf 

http://www.ohe.state.mn.us/tPg.cfm 

?pagelD=795&1534-D83A_1933715A= 

ccce61e27fb22fa2 



Linking state data 

Minnesota assigns all of its students a unique 
statewide student number, which is used to link 
student- level records across the state’s databases. 
Likewise, the state creates unique identifiers for its 
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teachers. At this time, Minnesota does not match 
records across teachers and students. 

After successfully applying for a Statewide Longi- 
tudinal Data System grant, Minnesota is par- 
ticipating in the U.S. Department of Education’s 
Statewide Longitudinal Data System program 
from November 2005 to October 2007. Minnesota 
is collaborating with Michigan and Wisconsin to 
build a comprehensive multi-state longitudinal 
data system. For additional information see: 

• http://www.dataqualitycampaign.org/survey_re- 
sults/state.cfm?st=Minnesota 

• http://nces.ed.gov/Programs/SLDS/PDF/Minne- 
sota.pdf 

• http://nces.ed.gov/Programs/SLDS/PDF/minneso- 
taabstract.pdf 

Data quality assurance 

The Minnesota Department of Education Data 
Center is responsible for data quality for the state. 
The Center reviews data to make sure more than 



one student is not assigned the same identifier or 
that students receive different unique identifiers 
when they change districts. Likewise, the Center 
works to prevent two teachers from being assigned 
a single identifier and to ensure teachers do not 
receive more than one identifier. 

Minnesota relies on separately collected and 
verified databases for its student demographic 
information. In addition, the state has a state- 
wide audit system to review data submitted. 
Statistical checks are also performed on the data 
submitted by the districts. If data are flagged by 
the statistical checks, Minnesota has established 
a system to investigate the data in question. 
Furthermore, the state has produced criteria 
for determining when data submitted by school 
districts are likely to be in error. On-site quality 
checks are an additional measure used by the 
state to promote accuracy in the data submitted 
by school districts. When school districts are 
found to be doing a poor job collecting and sub- 
mitting information or accounting for missing 
students, the state imposes consequences on the 
school districts. 
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Information needs assessment 

Data collection and quality assurance procedures 

The Educational Management Information 
System in the Ohio Department of Education is 
used to collect and maintain education data for 
the state. School district data are collected in 
regional data acquisition sites, called Informa- 
tion Technology Centers (ITCs), where data are 
cleaned and checked for errors. These sites are 
seen as the “first line of defense” prior to the 
data being sent to the state. Once submitted, 
data are processed and preliminary reports are 
posted, during which time districts can make 
changes to their data if there are problems or 
issues with reporting. 

A number of state-level offices contribute to the 
quality control process. Data are audited by a 
cross- agency team that constantly monitors the 
state’s data auditing processes and makes recom- 
mendations for improving them. Program offices 
are responsible for checking data related to their 
programs. A data services office has responsibility 
for quality control of the data. The state also relies 
on districts to monitor their data. Most major 
cities are their own data acquisition sites. Given 
the size of such districts and the volume of data 
they process, it is recognized that their submis- 
sions are particularly vulnerable. To assist districts 
in ensuring the quality of the data they submit, 
the state distributes numerous reports to districts 
and provides opportunities to verify information 
currently in the system and update information to 
correct any inaccuracies. 

State education agency officials noted no staffing 
needs with respect to data collection or quality 
control at the state level. Data reporting is accom- 
plished via a recently updated web site. 



Defining state data needs 

How Ohio defines data needs 

Ohio’s educational data needs are defined through 
a combination of statutory requirements and input 
from numerous Ohio Department of Education 
(ODE) offices and stakeholders. ODE offices pro- 
vide input on information that would be particu- 
larly valuable on specific topics (e.g., the curricu- 
lum office with respect to course offerings; the 
office of policy and accountability with respect to 
policy evaluations and program evaluations; and 
the offices that comprise the Center for Students, 
Eamilies, and Communities with respect to state 
reading funds, data related to discipline including 
‘persistently dangerous schools’, and school readi- 
ness and early learning). Such offices frequently 
have specific data needs, given the variety of chil- 
dren they serve and their unique characteristics, 
and participate in both formal and informal needs 
definition processes. Formally, individual centers 
meet annually to review information require- 
ments, including those associated with legislative 
mandates. More informally, data managers meet 
periodically with senior officials and other state 
education agency staff to continuously monitor 
data requirements. In addition, the department 
seeks input from a variety of stakeholders around 
the state. Formal mechanisms for obtaining stake- 
holder input include the Educational Management 
Information System (EMIS) Advisory Committee, 
a national group that advises the department. 
Together, participants in these processes work 
not only to identify information needs, but also 
to identify potential redundancies and whether 
information currently collected is underutilized. 

Information and data priorities 

Ohio’s goals in using educational data were 
described as twofold: first, to improve student 
learning (overall, and student learning of specific 
important content), and second to improve the 
effectiveness and efficiency of the educational sys- 
tem. Specific educational data goals are prioritized 
to obtain the information critical for determining 
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how to comprehensively address student needs. 
Such goals include: 

Supporting collaboration among the various 
education offices. For example, reading test scores 
may be examined in conjunction with student 
discipline and absenteeism information. 

Collecting data to address longitudinal questions 
(e.g., enabling evaluations of the long-term impact 
of curricula such as Reading First). 

Linking student with teacher data more easily. 

A cross-cutting concern, described as one of Ohio’s 
most pressing information needs, is to continue 
ongoing work to make data accessible to educators 
in ways that are actionable. This includes present- 
ing data in ways that do not require intended data 
users (e.g., administrators and teachers) to be data 
experts — making it easier for users to understand 
the implications of data for student learning, 
teacher professional development, and other out- 
comes. Key goals here are to ensure the right data 
are delivered to particular users in a timely way 
and in formats that are understandable (e.g., creat- 
ing more ‘friendly’, facile data interfaces). 

Establishing benchmarks. Ohio has already 
established benchmarks to measure the perfor- 
mance of various sub-groups and mechanisms 
to examine how Ohio is performing relative to 
the rest of the nation (e.g., using the National 
Assessment of Education Progress, NAEP). State 
education agency officials are also interested in 
benchmarking the state’s progress internation- 
ally, to ensure the state’s students are prepared 
to compete in the global economy. Another area 
of interest is providing benchmarks to assess the 
relative cost-effectiveness of particular strategies 
or interventions. 

Tracking progress. As the state’s student-level infor- 
mation system matures (at the time of writing the 
state had entered the third year of the collection 
of individual-level unidentifiable student data), 
additional information will become available for 



addressing longitudinal questions about student 
progress through the educational system. 

Ohio is interested in additional information on 
the utilization and impact of teacher professional 
development opportunities. A large number of 
professional development efforts are under way 
across the state, but it can be challenging to obtain 
good information not only about who attended 
what sessions, but also about the content and 
quality of workshops and other TPD activities. 
Particularly challenging here are developing and 
consistently applying definitions and measures of 
what constitutes high quality professional develop- 
ment. Currently, the state uses the STARS system 
to collect data on the types of professional devel- 
opment teachers pursue. Although this system 
collects information on what courses and work- 
shops teachers participate in for their professional 
development, no data are collected on how much 
teachers learn and use in their classrooms. 

Obtaining more information on transitions to post- 
secondary education was described as a recurring 
rather than an urgent need. Limited information 
is already available for students who attend Ohio 
public postsecondary institutions, but very little 
for those who attend private institutions in-state 
or higher education institutions out-of-state. A 
critical impediment to obtaining such data are 
legal prescriptions against sharing individual-level 
data outside the state system; the logistical issues 
encountered in tracking students as they leave high 
school are also challenging. Although Ohio uses 
a unique student identifier in the K-12 school da- 
tabase, a unique identifier is not used to link K-12 
student data to postsecondary student data. 

Improved processes for following students who 
transfer to community or alternative schools, 
and mechanisms for following the progress of 
children who attend preschool programs as they 
move into the elementary education system would 
also be beneficial. Ohio is also considering adding 
variables to the state report card to make it more 
informative, including measures of school climate 
and family engagement. 
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Understanding state data challenges 

This study identified multiple positive outcomes 
associated with the educational data system 
Ohio has put into place and continues to develop. 
Within the ODE, data have been used to promote 
more cross-center collaborative efforts to address 
student needs. For example, a monthly perfor- 
mance council was described as having evolved 
into a learning community which uses educational 
data to take a much more holistic and sophisti- 
cated approach to understanding the factors that 
affect student learning outcomes. Data are used 
throughout the system for continuous improve- 
ment and continuous learning. Ohio’s educational 
data have been used to provide information about 
where districts and schools need to be more stra- 
tegic in examining what they do and how they can 
improve their results. More generally, the state’s 
educational data have been described as creating a 
transparency and openness that have proved to be 
powerful catalysts for individuals to examine their 
practices and question whether they are achiev- 
ing the results they are capable of attaining, and 
for larger, positive, very constructive discussions 
about the results of the education system. These 
notable achievements notwithstanding, the state 
has identified additional challenges which it seeks 
to address. 

Ohio is working to eliminate problems associated 
with inconsistencies in data definitions across data 
sets, including the creation of a data warehouse 
that will enable datasets to work across each other 
more efficiently. Other inconsistencies and issues 
that can often prove challenging for large data 
collection and information processing systems 
were not characterized as problematic for the ODE 
(e.g., inconsistencies in data definitions over time, 
in coding procedures across data sets, items, and/ 
or over time, or in the intervals between data col- 
lections; incomplete or inadequate documentation 
of data sets; or problems resulting from data entry 
errors). 

More challenging is the inability to match records 
or link data sets to take advantage of information 



already collected at the state level. The difficulties 
here are not technical; the challenge here is a legal 
one, resulting from what has been described as the 
very restricted view federal attorneys have taken 
of states’ ability to share individual-level data col- 
lected in the K-12 system. Some systems have been 
developed to allow data sharing that does not em- 
ploy individual-level data, but there are limitations 
to additional efforts that can be made to leverage 
existing information absent a less restrictive rul- 
ing from the U.S. Department of Education. 

Other challenges are associated with the burden 
additional data collection can pose, including at the 
local level. Concern was expressed that informa- 
tion needs such as those described above may be 
difficult to achieve if additional resources cannot 
be provided to districts (e.g., to provide additional 
staff resources). At the state level, it was noted that 
it can be difficult to retain technical staff given sal- 
ary differentials with the private sector. 



State data inventory 

Ohio's educational data system 

The Ohio Department of Education’s (ODE) 
organizational structure consists of several offices 
responsible for managing the state educational 
data. Table G1 provides contact information for 
ODE staff involved in the educational data system. 
The infrastructure for collecting and maintain- 
ing the data are managed by the Office of Infor- 
mation. This office maintains and updates the 
department’s technology tools, including com- 
puter hardware, software, and Web applications. 
Instructions for school districts on collecting data 
on student performance and school district needs 
are provided by the Office of Data Services, which 
develops the Education Management Information 
System (EMIS) manual. There are four types of 
data in the EMIS: student data, staff data, district 
and building data, and financial data. When the 
state department receives data from the school 
districts, the Information Technology Centers 
(ITC) will process the data by extracting, aggre- 
gating, validating, formatting, compiling, and 
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TABLE G1 

Key contacts for more information on Ohio's data processes and files 



These data collection activities ... ... are currently the responsibility of: 



Education Management Information System (EMIS) 


Beth Juillerat, Director of Data Services, 614-752-8368 


EDEN Coordinator 


Unknown 


Ohio Department of Education (Operations) 


Steven Burigana, Chief Operating Officer, 614-466-3763 


Ohio Department of Education (Information) 


Amy Ancles, Chief Information Officer, 614-466-3763 


Ohio Department of Education (Technology) 


Gregory Davidson, Chief Technology Officer, 614-387-0339 


Application development 


Michael Carmack, Director, 614-466-5440 


Project Management 


Wendi Boggs, Director, 614-995-5928 


Technical Services 


Brian Brown, Director, 614-644-0706 


transferring it. ITC also includes the Ohio Educa- 
tion Computer Network (OECN), which collects 
and aggregates the EMIS district data. 


• Data Quality Campaign web site, http://www. 
dataqualitycampaign.org/survey_results/state. 
cfm?st=Ohio. 


Contact information is collected and maintained 
by systems in ODE. A directory of educational 


• Ohio Department of Education web sites: 


institutions is maintained by the Ohio Education 
Directory System (OEDS). The Ohio Educational 
Directory System Redesign (OEDS-R) is a decen- 
tralized data system that allows organizations 


• http://www.ode.state.oh.us/GD/Tem- 
plates/Pages/ODE/ODEDetail.aspx?page= 
3&TopicRelationID=17&Content=17172. 


(e.g., public districts, public schools, community 
schools, nonpublic schools, etc.) to maintain their 
own data. OEDS is accessible to the public for the 
purposes of obtaining the most current informa- 


• http://www.ode.state.oh.us/GD/Tem- 
plates/Pages/ODE/ODEDetail.aspx?page= 
3&TopicRelationID=60&Content=18212 


tion about an organization. 
Key data elements 


• Education Data Exchange Network (EDEN): http:// 
www.ed.gov/about/offices/list/oig/auditreports/ 
alle0003.pdf) 


The Data Quality Campaign conducted a survey 
to determine which key data elements each state 
in the country has in its educational data system. 
Based on their survey, Ohio has data elements 
related to student demographics, student special 


• Information about enrollment data for public and 
nonpublic school districts. http://www.ode.state. 
oh.us/GD/Templates/Pages/ODE/ODEDetail.aspx? 
page=3&TopicRelationID=396&Content=17431. 


population status, student level enrollment data, 
student course taking, student achievement, 
student special program participation, student 
college-readiness test scores, and student gradua- 


• Statewide Longitudinal Data System program, 
conducted by U.S. DOE. http://www.ed.gov/pro- 
grams/slds/index.html 


tion and dropout data. 


Student data 


The remainder of this section provides an over- 
view and highlights of student and teacher data 
currently collected and archived by the state of 
Ohio. For additional information see: 


Demographics. Student demographic data ele- 
ments include: gender (collected first in 2000/01), 
date of birth, age, ethnicity (collected first in 
2000/01), race (collected first in 2000/01), and 
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low-income status (collected first in 2002/03). All 
of these elements are contained in the state-wide 
student demographic record included in EMIS. 

Also included in the student demographic record 
is information about the special population status 
of students. These data are collected with the 
same frequency as the demographic data and are 
included in the student demographic record. These 
elements include English proficiency status (first 
collected 2001/02), disability status (first collected 
2000/01), economic disadvantage status (first col- 
lected 2002/03), and migrant status (first collected 
2001/02). 

Enrollment. Student enrollment data elements 
include: school in which the student is enrolled, 
student mobility (first collected 2001/02), and 
student attendance data. If a student has been 
enrolled in more than one district during the 
reporting period, each district submits a student 
demographic record. 

Student mobility and student attendance are both 
contained in the Student Attendance Record in- 
cluded in the Education Management Information 
System. EMIS collects these records at the end of 
specified reporting periods, and manages the data. 

Course-taking. Information regarding student 
course-taking is contained in the Student Course 
Record, which is included in the EMIS. A separate 
student course record is reported for every course 
in which the student is participating, and the state 
uses a statewide system for coding the courses. 
Course completion data include middle school 
courses taken for high school credit, summer 
school courses taken for high school credit, and 
dual enrollment courses taken from college and 
universities. 

Special program participation. The Student 
Program Record contains information regarding 
student special program participation, including: 
early childhood program participation, individu- 
alized education program participation, special as- 



sistance programs, and honors programs/courses. 
The Student Program Record is part of the EMIS. 

Achievement. The state of Ohio collects student- 
level test data annually. These data include the 
Ohio Achievement Test record, the Ohio Profi- 
ciency-Only Test record, the Ohio Ninth-Grade 
Proficiency Test record, the Ohio Graduation Test 
record, the Preschool Assessment record, the 
Ohio test of English Language Acquisition record, 
and the CTAE Student Assessment record. These 
test records are all stored in the EMIS. The Ohio 
Achievement Test record data are reported during 
the year-end reporting period, and test records are 
reported for all students enrolled during any test 
administration in the current school year. These 
tests include reading, math, science and social 
studies from 3'’’^ grade to S*** grade. Test data are 
stored permanently by the state, and the state does 
collect and maintain individual records of each 
untested student in a tested grade. Student-level 
SAT or ACT exam results are not collected by the 
state. 

Graduation and dropout data. Ohio collects data 
on graduation status in the Student Attendance 
Record. These elements include the graduation 
credit units and retained status. These data are 
collected and managed by the EMIS. Information 
on diplomas is found in the “diploma type ele- 
ment” in the Student Attendance record. The state 
collects annual dropout data and stores it in the 
Student Attendance record in the EMIS. The state 
also records data on the “Withdrawal/Dropout/ 
Truancy Reason element” which provides “the 
reason for the most recent withdrawal from the 
school district.” 

Teacher data 

Demographics. Information on teacher de- 
mographics is collected annually and is 
found at http://www.ode.state.oh.us/GD/ 
DocumentManagement/DocumentDownload. 
aspx?DocumentID=12577. This information is 
stored in the EMIS, and EMIS is responsible for 
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collecting, maintaining and providing access to 
this data. These data were first collected in 1999. 

Certification, qualifications, and professional 
development. Teacher certification, qualification, 
and professional development information are 
collected annually and can be found in the iLRC 
(interactive Local Report Card) at http://ilrc.ode. 
state.oh.us/Downloads.asp. The iLRC collects, 
maintains, and provides access to these data, 
which were first collected in 2001/02. 

Mobility and attrition. Teacher mobility and 
teacher attrition information are collected an- 
nually and stored in the Education Management 
Information System (EMIS), found at http://www. 
ode.state.oh.us/GD/DocumentManagement/ 
DocumentDownload.aspx?DocumentID=12577. 
EMIS is responsible for collecting, maintaining, 
and providing access to this data and has collected 
these data elements since 1999. 

Linking state data 

Ohio was awarded a $5.7 million grant in Novem- 
ber of 2005 to build a statewide longitudinal data 
system. Efforts to build this system in Ohio are 
contingent on the ability to link data. Ohio has 
successfully implemented the Statewide Student 
Identifiers (SSIDs) system. A third party vendor 



assigns the unique SSID, which is a nine character 
identification code for each public school student 
within the Ohio public education system. The 
SSID is used for EMIS reporting purposes to retain 
privacy, and may not at any time be cross-walked 
with the data files that contain personal student- 
level data. Teachers are also assigned a unique 
statewide teacher identifier. 

Data quality assurance 

Ohio has several processes for ensuring the 
accuracy of its educational data. Districts are en- 
couraged to review their data for accuracy before 
they submit data to the state via EMIS. On-site 
quality checks, however, are not conducted at the 
schools. When the state receives the district data 
from EMIS, statistical checks are performed and a 
system is used determine the accuracy of the data. 
Consequences are enforced on districts if their 
data are not validated by the state data quality as- 
surance processes. 

The state of Ohio currently has procedures in place 
to prevent two different individuals from receiving 
the same ID and also to prevent the same student 
from receiving a different ID when she/he changes 
districts. Ohio also has policies to ensure that 
teachers do not receive different IDs when they 
change districts. 
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Information needs assessment 

Data collection and quality assurance procedures 

In Wisconsin, the Department of Public Instruc- 
tion (DPI) is responsible for much of the state’s 
educational data collection, with the Information 
Technology (IT) team of the Department respon- 
sible for meeting the technical aspects of the state’s 
data needs. The state occasionally contracts for 
various collections as well. 

Data are audited in part by IT staff with help from 
DPI program experts, although this department 
believes that more auditing at the state and local 
level are important to improve data quality. DPI 
is working to maximize the efficiency of auditing 
efforts but resource constraints, growing federal 
mandates, and shorter federal time lines for sub- 
mitting required data limit progress on this task. 
Automated validity logic checks are performed to 
look for missing data, invalid codes, and inconsis- 
tent relationships with other data in the system. 
The state education agency also checks for possible 
problems or areas where data quality problems 
may exist. If the state finds problems that are 
not picked up by the various groups submitting 
data, the state communicates with them about 
the issues and explains what might be happening 
and what might be wrong. The state also performs 
some site visits to determine if there are issues 
with data quality. 

After data are collected, data are reported to the 
public and other stakeholders via the Wiscon- 
sin Information Network for Successful Schools 
(WINSS) web site. WINSS integrates data from 
a wide range of DPI data collections to provide 
a web-based version of mandated reports. But, 
more importantly, WINSS facilitates the use of 
data included in mandated collections for school 
improvement purposes by providing flexible 
graphing and download tools as well as software 
to facilitate the collection and use of optional 



data. Links on WINSS graph pages explain where 
the data come from, limitations, and how rates 
and other indicators are calculated. Much of the 
information presented on this web site is driven by 
various federal programs, including EDEN, NCLB, 
and IDEA. More information about WINSS is 
provided at http://dpi.wi.gov/sig/ppt/winssdata. 
ppt. A site map is available at http://dpi.wi.gov/ 
sig/sitemap.html. The Longitudinal Data System 
project will integrate and enhance WINSS provid- 
ing local educators with access to more data and 
more tools. 

DPI has many separate data collections and is 
working to integrate these collections to more ef- 
ficiently meet growing reporting mandates and to 
provide more useful information at the same time. 
Data collection goals and procedures, including 
various codes and data elements are documented 
on DPI web pages. The key student level collec- 
tion is the Individual Student Enrollment System 
(ISES). Information about ISES is provided at 
http://dpi.wi.gov/lbstat/isesapp.html. The WINSS 
web site does not provide information about how 
data are collected, because it is a reporting/presen- 
tation site not a collection site. 

Wisconsin also has an ongoing data dictionary 
project. The department has been working to obtain 
consistent data definitions across program areas 
for a decade or more and has made intermittent 
progress. Consistent long-term commitment of staff 
and resources to this task would facilitate work as 
this is an on-going task and a more comprehensive 
dictionary is needed for the integration of collection 
and reporting systems. DPI has hired a limited term 
employee with experience in the creation of data 
dictionaries to facilitate the planning process, but 
stair and money are likely to be continuing barri- 
ers to maintenance and evolution of the dictionary. 
This data dictionary project is described more as a 
business task than a technical task. 

While there is a person whose job responsibilities 
broadly include being the “data quality coordina- 
tor,” Wisconsin does not have designated staff 
at the state level whose sole responsibilities and 
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authority are related to monitoring data quality. 
Although there are many conversations about data 
quality at the state level between individuals work- 
ing on various collections or aspects of data usage, 
personnel working on individual data collections 
maintain primary responsibility for the quality of 
those collections while at the same time juggling 
other responsibilities. 

Defining state data needs 

How Wisconsin defines data needs 

Wisconsin’s data needs are defined by many peo- 
ple, including several groups who analyze changes 
in state and federal laws to determine how the data 
collected and reported need to change to meet new 
regulations and rules. Formally, there is a data 
management steering committee that becomes 
involved when new data needs require changes in 
long-term plans, priorities, or additional re- 
sources. There are also other more “ad hoc” groups 
that assemble from various divisions when there 
are changes in requirements or new requests that 
do not fit neatly into any existing group. Usually, 
the IT section that is responsible for the technical 
aspects combines with people from program areas 
who need the data to determine collection and 
reporting needs, define data elements and collec- 
tion categories, create automated business rules 
and warnings, and generate reports. 

Wisconsin also tries to involve as many stakehold- 
ers as possible in defining data requirements and 
needs. Ideally schools, communities, advocacy 
groups, other state agencies, and other govern- 
mental jurisdictions are involved, all of whom 
might be interested in using the data and relating 
it to their own information. Wisconsin consid- 
ers the interests of these groups when defining 
state data needs. Developing and implementing 
plans to maximize access to, and usefulness of, 
data for all stakeholder groups is a DPI goal (see 
Guiding Principles http://www.dpi.wi.gov/lbstat/ 
isesprinc.html), but one that is difficult to achieve 
due to growing federal mandates and state/district 
resource constraints. 



Information and data priorities 

A key goal for Wisconsin in terms of using edu- 
cational data is to make it available and accessible 
to educators and other stakeholders who work to 
improve student achievement and the educational 
experience. Beyond understanding student needs, 
Wisconsin wants to help people understand which 
courses of action might help address those needs 
and improve outcomes. This includes developing 
ways to present data that will assist in determining 
whether students are making progress, and ideally 
to connect interventions with student achievement. 
Such work can only be achieved in the context of re- 
porting mandates, a primary goal of the department. 

Wisconsin’s data currently can address questions 
about student progress and achievement, and 
make comparisons across groups. The state also 
would like to be able to link interventions with 
student progress and improvement, but DPI cur- 
rently does not collect data on interventions. Some 
Wisconsin educators would like to make inter- 
vention data an optional submission so that local 
educators can more efficiently discuss the effec- 
tiveness of past interventions and make on-going 
adjustments accordingly. 

The remainder of this section reviews the kinds of 
questions the state of Wisconsin would most like 
its educational data to address in the short-term 
and looking further to the future, and highlights 
the information needs that senior state education 
agency officials currently consider to be particular 
priorities. Existing challenges which may affect 
the state’s ability to achieve these objectives mov- 
ing forward are discussed below. 

Establishing benchmarks. Comparing Wisconsin’s 
progress with other states and with the nation 
overall is of interest to the agency if comparable 
data are readily available. A high priority question 
for Wisconsin, and one that is currently being ad- 
dressed, is how various subgroups or subpopula- 
tions within the state are performing. The agency 
has been aggregating and disaggregating data by 
population subtype for the last ten years. 
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Tracking progress. Wisconsin began collecting 
grade-level enrollment data with unique student 
identifiers in fall 2004 as part of the Individual 
Student Enrollment System (ISES). This system 
tracks movements of Wisconsin students from 
school to school and district to district and is used 
to identify students as dropouts or graduates. 
Implementation of ISES continues to be a major 
challenge for DPI and school districts because, 
until 2004, Wisconsin did not have a statewide 
student level data collection and students did not 
have state identifiers. 

A current priority is obtaining information on 
postsecondary transitions, although this may not 
be feasible for several more years given other more 
pressing needs coming from EDEN and IDEA. 
While the state would like more and better infor- 
mation on a variety of topics, many of its resources 
are currently being expended to provide informa- 
tion required under federal law. 

A major resource constraint for collecting and 
reporting postsecondary or other data is the need 
for more staff, especially in the IT area. Even if 
money is available to hire consultants, agency 
stair must be available to train consultants on 
operations within the department and to main- 
tain and modify systems and tools developed 
by consultants. Consulting in IT might still be 
helpful, however, in terms of developing more 
efficient ways of connecting, reporting, and using 
information. 

Wisconsin collects some curriculum data about 
course offering and course taking, but these data 
are not always reliable or complete. With respect 
to curriculum data needs, the state has created a 
resource called the Curriculum Resource Center 
(http ://www2 . dpi. state.wi.us/sig/practices/high_2 . 
asp) that provides access to thousands of high 
quality MarcoPolo classroom activities, lesson 
plans, etc. aligned with national standards. The 
department spent considerable effort in the recent 
past aligning MarcoPolo lesson plans, etc., with 
state standards, which enables teachers to pull up 
lessons in specific subject content areas by grade 



level. Currently, it is challenging to update the 
database regularly given personnel constraints. 
Such staffing issues are not easily solved by outside 
hiring since knowledge of the Wisconsin academic 
standards is required. 

Documenting outcomes and their causes. Wiscon- 
sin is interested in documenting specific outcomes 
related to areas such as academic achievement, 
drop out rates, and transitions to postsecondary 
education. Many of these types of questions have 
already been addressed and are reported on the 
WINSS web site. Obtaining data on transitions to 
postsecondary education, while of high interest, 
presents more of a challenge. Wisconsin now only 
collects aggregate data on post-graduation inten- 
tions, and limited surveys are conducted to collect 
more information as required. 

Another area of interest is to find ways to monitor 
outcomes more frequently and more locally for 
teachers and school staff. Ideally, the state would 
like local educators to use on-going formative 
assessments so that teachers can actually adjust 
lessons as learning is happening rather than at the 
end of the year. This sort of system might be de- 
signed more as an optional resource for educators, 
and be incorporated into tools for instructional 
teams. Wisconsin also seeks to answer questions 
about how to use time and money more efficiently 
in districts, schools, and classrooms. 

Enhancing capacity to use existing data resources. 
Additional staff and long-term funding would 
allow Wisconsin to develop and maintain systems 
for using data more effectively. Training in data 
use at the state level is currently sporadic and 
informal due to a lack of staff time and resources. 

Understanding state data challenges 

Wisconsin’s educational data system already has 
made important impacts. There is less reliance on 
anecdotal information and more reliance on what- 
ever factual information is available to help guide 
discussions and focus plans. A variety of stake- 
holders are developing a genuine interest in having 
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and using education data, and the state is working 
to provide information necessary to make better 
informed decisions. However, current data collec- 
tion and reporting requirements more than use up 
available resources. The state is working to meet 
competing deadlines and must reprioritize work 
on a daily basis. Although agency staff are well 
aware of issues associated with collecting, report- 
ing, and analyzing data, it can be challenging to 
attend adequately to all of them. Staff turnover is a 
challenge as well since departing permanent staff 
generally have no backups and funds may only be 
available to hire temporary employees. Addition- 
ally, it would be helpful if the state received as- 
sistance in data quality and analysis since current 
staff are focused on data collection requirements 
and assisting districts with required reports. 

Wisconsin seeks to have faster turnaround of state- 
wide test data and for that data to provide more 
information about students’ needs within a subject 
area. Other challenges facing the state include: 

• There is a lack of time and staff to meet federal 
reporting requirements even as these continue 
to grow. Wisconsin is working to address 
these challenges and has made substantial 
progress (e.g., adding a unique statewide 
student identifier — with invaluable support in 
the planning stages from ESP solutions — and 
implementing its first statewide student level 
data collection.). Concerns were expressed, 
however, that additional requirements can 
overtake whatever progress has been made. 
The state would benefit from a secure funding 
stream to hire permanent staff able to main- 
tain new systems (once grants have expired). 

• Inconsistencies in data definitions present 

a problem for Wisconsin, although the state 
has made substantial progress in eliminating 
such inconsistencies. A comprehensive data 
dictionary is needed. DPI has hired an LTE to 
assist with the planning process. Plan imple- 
mentation and maintenance of the dictionary 
are likely to suffer from insufficient staff and 
other resources. 



• Labels, definitions, and collection/reporting 
categories are often modified to comply with 
changes in the law or for other reasons. Such 
modifications can cause confusion, data quality 
problems, and the loss of some longitudinal 
information not to mention the expense of 
reprogramming. Wisconsin is working to allevi- 
ate this problem by making stakeholders aware 
of the ramifications involved in making these 
types of modifications. Resources that have 
been helpful in this regard include the national 
Data Quality Campaign and the EDEN initiative 
which promotes consolidation of data collection 
requirements across federal programs. 

• Inconsistencies in coding procedures are a 
recurring problem as changes in data col- 
lection and/or reporting requirements may 
necessitate adding or splitting codes. The state 
is aware of these issues and addresses associ- 
ated problems. 

• Like other states, Wisconsin has experienced 
problems coordinating the collection of re- 
lated information as state and federal agencies 
often have conflicting deadlines for collecting 
required data and may request very similar 
types of data. 

• Wisconsin is fairly consistent in its treatment 
of missing data but has more missing data than 
the department would like, often because dis- 
tricts are unable to meet reporting deadlines. 
The state made the transition from having al- 
most all aggregated data to student level data in 
a very short period of time, and some districts 
are still struggling with this transition even as 
timelines become increasingly rigid. 

• Data entry errors are a problem for the state. 
Wisconsin continues to develop automated 
systems that are able to flag data entry errors; 
however, schools are burdened with respond- 
ing to error and warning messages that the 
system produces in order to complete col- 
lections, which in turn creates a burden for 
agency help-desk staff. 
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• Wisconsin makes every effort to meet report- 
ing requirements and requests for data from 
individuals who want to use it for studies or 
publications, although the agency cannot 
always respond to more informal requests 
in a timely fashion. Wisconsin has made a 
wide range of redacted data files available 
for download. The Data Quality Campaign’s 
initiative to coordinate data requests across 
partner organizations has helped reduce the 
number of data requests. Concerns remain 
about maintaining privacy in accordance with 
state and federal laws. 

Opportunities and challenges of linking data 

Wisconsin has made substantial progress in link- 
ing data and matching data sets, particularly via 
the recent implementation of a statewide student 
numbering system. The department has also cre- 
ated a consistent list of schools with a consistent 
set of codes across program areas in order to fa- 
cilitate the matching of databases and records. As 
noted above, the data dictionary project is in the 
planning stages and would benefit from outside 
expertise and internal staff dedicated to this pro- 
cess. When completed, this data dictionary will be 
made publicly available. 



State data inventory 

Wisconsin's educational data system 

The Wisconsin Department of Public Instruction 
“collects a wide range of statistics and program 
data from the educational and library communi- 
ties to meet State and Federal legislative require- 
ments” (see http://dpi.wi.gov/lbstat/data.html). The 
introduction to the “Data” section of the DPI web 
site provides links to forms used in data collec- 
tion, “web sites for data submission,” “supporting 
documentation and instructions,” an “extensive 
array of reports on student performance, demo- 
graphics, special education, teachers and staff, 
finance, public libraries, and school performance” 
and other information (see http://dpi.wi.gov/lbstat/ 
data.html). 



All data collections authorized by the DPI are 
included in the Data Collection Plan, a “planning 
document and reference tool” that “lists recur- 
ring forms . . . authorized for collection to meet 
the needs of programs” that the DPI administers 
(see http://www2.dpi.state.wi.us/datacoll/StSupt. 
htm; http://www2.dpi.state.wi.us/datacoll/default. 
asp).^ At the time of writing, the DPI Data Collec- 
tion Plan provided information on 363 collections 
including those whose usage is internal (“used 
only within DPI”), local (“distributed by DPI for 
local use only and not returned to the DPI”), and 
external (“sent outside DPI to be completed and 
returned to the department”) (see http://www2. 
dpi.state.wi.us/datacoll/AboutPub.htm). Lists of 
sets of forms meeting specific search criteria (e.g., 
generated by keyword or primary respondent 
searches) provide key summary information for 
each collection, including: form number and title, 
due date(s); primary respondent; and the division, 
team/program, and names and telephone numbers 
of key DPI contacts. In many cases, links to elec- 
tronic versions of print- only forms and/or elec- 
tronic fillable forms are also provided; other data 
collections employ “full-blown web applications 
that use defined programming codes to activate an 
action.”^ 

The DPI Library and Statistical Information 
Center’s web pages are another source of detailed 
information on Wisconsin education data col- 
lections.^ From the Center’s home page, visitors 
to the DPI web site can access a “Data Collection 
Reporting Schedule” and descriptions of and links 
to other DPI Library and Statistical Information 
Center services and resources. Other valuable 
sources of information include DPI’s descriptions 
and documentation of: 

• The Individual Student Enrollment System 
(ISES), which enables districts and schools to 
provide the “student demographic and out- 
come data needed to meet the ESEA [Elemen- 
tary and Secondary Education Act] report card 
requirements at the state, district, and school 
levels” via file upload or on-line data entry 
(see http://dpi.wi.gov/lbstat/isesapp.html). 
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Wisconsin school districts began to participate 
in the ISES in the 2004-05 school year (see 
http://dpi.wi.gov/lbstat/ises_5ws.html). 

• The Wisconsin Student Number Locator 
System (WSLS) which is used “(1) to assign 
new Wisconsin Student Numbers (WSNs) to 
students entering Wisconsin Public Schools, 
(2) to help ensure that the WSNs stay with stu- 
dents as they move from school to school and 
district to district, (3) to update WSLS data 
such as when students exist a school or when 
data used for matching change (e.g., legal 
name, guardian), and (4) to correct errors in 
WSLS birthdates, spellings, etc.” (see http:// 
dpi.wi.gov/lbstat/wslsapp.html). 

• The Wisconsin Student Assessment Sys- 
tem (WSAS), which “includes both regular 
assessments taken by nearly all students 
and alternate assessments taken by certain 
students with limited English proficiency or 
disabilities” (see http://dpi.state.wi.us/oea/ 
wsas_intro.html). 

• The School Performance Report, including 
the May 2006 version of the 2005-2006 School 
Performance Report Data Definitions and 
Directions (available online at http://dpi.wi.gov/ 
spr/doc/sprdir06rev2.doc), and the May 2004 
version of the School Performance Report 
Glossary (online at http://dpi.wi.gov/spr/doc/ 
sprgl04.doc). 

Key data elements of Wisconsin’s educational data 
system are summarized and reported by the De- 
partment of Public Instruction on the agency web 
site at http://dpi.wi.gov/index.html. 

The DPI Library and Statistical Information 
Center’s home page provides links to information 
on the School Performance Report, and statistical 
data and reports available from the Department of 
Public Instruction Statistical Information Center 
(http://dpi.wi.gov/lbstat/statpg.html). In addition, 
most of the data collected by DPI are available on 
the Wisconsin Information Network for Successful 



Schools (WINSS) pages. WINSS is an electronic 
resource that has been created “to help educators, 
parents, and community members who have an 
interest in educating the hearts and minds of all 
children. The Standards and Assessment, Data 
Analysis, Continuous School Improvement, and 
Best Practices sections guide users to key local, 
state, and national information about success in 
education as found on the DPI web site and other 
valuable resources” (see http://dpi.state.wi.us/sig/ 
index.html). This web site was created through 
a partnership of the North Central Regional 
Educational Laboratory (it was noted that without 
the NCREL partnership, WINSS would not exist), 
the Office of the Governor, and the Department of 
Public Instruction (DPI). 

WINSS data files are created using data stored (in 
Oracle tables) at DPI. There are five student demo- 
graphic elements, four student special population 
status elements, four student-level enrollment 
data elements, one student course-taking ele- 
ment, three student achievement elements, three 
student special program participation elements, 
two student college-readiness exam score ele- 
ments, and three student graduation and drop 
out elements. WINSS also provides information 
on teacher certification and qualifications and 
some information related to teacher professional 
development. 

Additional information on ISES, WINSS, WSAS, 
WSLS, the work of the Division for Libraries, 
Technology, and Community Learning (including 
the Library and Statistical Information Center), 
and other key data reporting activities is available 
online, and from the individuals listed in table HI. 

Key data elements 

This section provides an introduction to student 
and teacher data currently collected and archived 
by the state of Wisconsin. Additional information 
on data collection and reporting of specific data 
elements is available online (see the listing of data 
elements included in the ESEA Data Site Map, at 
http://dpi.wi.gov/lbstat/eseamap.html). 
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TABLE HI 

Key contacts for more information on Wisconsin's data processes and files 



These data collection activities ... ... are currently the responsibility of: 



1 These data collection activities . . . 


... are currently the responsibility of: | 


EDEN Coordinator 


Tiffany Boyd 

(608) 267 9162 

tiffany.boyd@dpi.state.wi.us 


Individual Student Enrollment System (ISES) 


Tiffany Boyd (technical and support) 
(608) 267 9162 
tiffany.boyd@dpi.state.wi.us 

Jean Whitcomb (content) 

(608) 266-2937 

jean.whitcomb@dpi.state.wi.us 


Wisconsin Department of Public Instruction, Division for 
Libraries, Technology, and Community Learning 
Library and Statistical Information Center 


Richard Grobschmidt 
Assistant Superintendent 
(608) 266 2205 

richard.grobschmidt@dpi.state.wi.us 

Roslyn Wise 

(608) 266 6439 

Roslyn.wise@dpi.state.wi.us 

Kay Ihlenfeldt 
Team Leader 
(608) 266 3108 

kay.ihlenfeldt@dpi.state.wi.us 
Barb Ballweg 

Education Statistics Coordinator 
(608) 266 1730 

barbara.ballweg@dpi.state.wi.us 


Wisconsin Information Network for Successful Schools 
(WINSS), Data Analysis 


Jean Whitcomb 
(608) 266-2937 

jean.whitcomb@dpi.state.wi.us 


Wisconsin Student Assessment System (WSAS) 


Office of Educational Accountability 
oeamail@dpi.state.wi.us 


Wisconsin Student Number Locator System (WSLS) 


Jean Whitcomb 
(608) 266-2937 

jean.whitcomb@dpi.state.wi.us 


Student data 


of the Rehabilitation Act”; see http://dpi.wi.gov/ 
Ibstat/datawsas.html). Also available online 


Demographics. Public school districts supply 
WSLS with key demographic information, in- 
cluding student gender, race, and birth date (see 
http://dpi.wi.gov/lhstat.datawsn.html). Additional 
demographic and student special population 
status information is collected via the ISES, 
including beginning in fall 2005 a ‘long-term US 
student indicator,’ migrant status, and Section 
504 status (to indicate students with “a physical 
or mental impairment covered under Section 504 


is detailed information on data collection and 
reporting of: racial/ethnic group and gender (see 
http://dpi.wi.gOv/lbstat/dataracgdr.html#gender); 
economically disadvantaged status (see http://dpi. 
wi.gov/lhstat/dataecon.html); English language 
proficiency (see http://dpi.wi.gov/lbstat/ddataelp. 
html), and the “major or overriding disability 
condition that best describes an IDEA-eligible 
student’s impairment” (primary disability; see 
http://dpi.wi.gov/lbstat/datadisab.html). 
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Enrollment. The Individual Student Enrollment 
System (ISES) replaced the PI-1290 Fall Enroll- 
ment Collection in 2004/05 (with counts of 
economically disadvantaged students based on 
the October DPI School Lunch Collection and 
counts of English language learners based on the 
LEP data collection through 2004/05). Prior to the 
2004/05 school year, enrollment counts were based 
on data obtained in the PI-1290 Fall Enrollment 
Collection (the October DPI School Lunch Collec- 
tion for counts of economically disadvantaged stu- 
dents, the LEP data collection for counts of English 
language learners, and the PI-2197 Federal Student 
Data Collection for counts by disability). Included 
in these counts are students enrolled in “all public 
schools required by law to submit School Perfor- 
mance Report data,” including “schools operated 
by districts . . . and nondistrict charter schools.” 

As “nearly all these requirements apply only to 
public schools” the Department of Public Instruc- 
tion “has very little data about private schools” 
(http://www.dpi.wi.gov/spr/demog_q&a.html). 
Available online is detailed information on data 
collection and reporting of: school enrollment and 
exit dates (see http://dpi.wi.gov/lbstat/dataenroll. 
html); grade level placement data collection and 
reporting (see http://dpi.wi.gov/lbstat.datagrade. 
html); the schedule of the educational program 
in which a student is enrolled (program schedule, 
see http://dpi.wi.gov/lbstat/datasched.html); at- 
tendance collection and reporting (see http://dpi. 
wi.gov/lbstat/dataattend.html); and the circum- 
stances under which students exit from schools 
(exit type; see http://dpi.wi.gov/lbstat/dataexit. 
html). 

Course-taking. Local school districts submit 
data on course olferings in a Course Offerings 
Report, also referred to as the Curriculum Re- 
port, (PI-1215); these data can be used to provide 
information on course taking patterns (see http:// 
dpi.wi.gov/spr/course_q&a.html). Information 
collected via the Course Olferings Report include: 
“the total number of different courses offered for 
Advanced Placement (AP), Cooperative Academic 
Partnership Program (CAPP), and SPR Advanced 



Coursework”; enrollments under listed course 
titles (including Advanced Placement courses, 
foreign languages, mathematics and science) and 
enrollments by subject area in CAPP programs al- 
lowing students “to take college-level courses and 
receive college credits”; foreign language instruc- 
tion offered in grades kindergarten through five; 
high school graduation requirements; part-time 
open enrollment; and youth options (see http://dpi. 
wi.gOv/apps/doc/1215inst06.doc). PI-1215 can be 
accessed (a print-only form can be downloaded) 
online via http://www2.dpi.state.wi.us/datacoll/ 
PInum.asp; 2005/06 instructions for completing 
the report are also available online (at http://dpi. 
wi.gOv/apps/doc/1215inst06.doc). 

Special program participation. The Special Educa- 
tion Team (in the Division for Learning Support: 
Equity and Advocacy) “collects and processes 
data on students in special education programs 
in response to federal requirements. This includes 
the unduplicated child count, implementation of 
FAPE [free appropriate public education] report, 
the student exiting report, the discipline and the 
personnel reports” (see http://dpi.wi.gov/sped/cc_ 
data.html). Both the ISES and the Federal Student 
Data Report (Child Count) collect student-specific 
data; “other state special education programming 
data comes from the Local Performance Plan sub- 
mitted by each school district in Wisconsin, the 
School Performance Report, and through surveys,” 
(http://dpi.wi.gov/sped/dm-speceddata.html). Data 
definitions for the IDEA Federal Student Data 
Report (PI -2197) for the 2006/07 school year are 
available online (at http://dpi.wi.gov/sped/doc/ 
ccinstr07-dd.doc). An ‘educational environment’ 
field (reflecting “the extent to which students with 
disabilities are educated with their nondisabled 
peers”) “will be collected on an optional pilot basis 
in 2006/07 ISES Count Date records in prepara- 
tion for possible consolidation of the PI-2197 
(Child Count) and WSLS/ISES in 2007/08.” The 
educational environment field “will be required in 
2007/08 ISES Count Date records for students with 
disabilities.” For additional information on this 
field, see http://dpi.wi.gov/lbstat/dataenvir.html. 
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Achievement. The Wisconsin Student Assessment 
System (WSAS) is one mechanism through which 
“students demonstrate their progress toward 
achieving the academic standards in English lan- 
guage arts, mathematics, science, and social stud- 
ies.” Currently the system “includes both regular 
assessments taken by nearly all students and 
alternative assessments taken by certain students 
with limited English proficiency or disabilities” 
(see http://dpi.wi.gov/oea/wsas_intro.html). 

From (spring) 1989 to 2005, third-graders’ read- 
ing comprehension was assessed by the Wis- 
consin Reading Comprehension Test (WRCT) 

(see http://dpi.wi.gov/oea/wrct3.html). Begin- 
ning in Fall 2005, Wisconsin Knowledge and 
Concepts Examination — Criterion-Referenced 
Tests (WKCE-CRT) replaced the WRCT and 
WKCE reading and mathematics tests. Student 
performance on the WKCE-CRT “is reported 
in proficiency categories and used to determine 
the adequate yearly progress of students at the 
school, district and state levels” (http://dpi.wi.gov/ 
oea/kce.html). Included in these standardized 
tests are “commercially-developed questions 
used in schools across the country and questions 
developed specifically for Wisconsin in order to 
improve coverage of Wisconsin academic stan- 
dards. The WKCE-CRT measures achievement 
in reading, language applications, mathematics, 
science, and social studies using multiple- choice 
and short answer questions. Students also provide 
a rough draft writing sample. Total WKCE-CRT 
test time varies by grade and may range from 5 to 
8.5 hours” (see http://dpi.wi.gov/oea/kce.html). 

The resulting information on how Wisconsin stu- 
dents are performing academically can be viewed 
graphically (and the associated raw data can be 
downloaded) via WINSS (e.g., http://data.dpi.state. 
wi.us/data/graphshell.asp?ORGLEVEL=ST&FUL 
LKEY=ZZZZZZZZZZZZ&DN=None-FChosen&S 
N=None-FChosen&GRAPHEILE=GEDISA). Other 
useful online sources for understanding and using 
WSAS data include the WINSS Similar Schools/ 
Districts tool (http://www.dpi.state.wi.us/winss/ 



similar.html), the WSAS Scatterplots (http://www. 
dpi.wi.gov/winss/kcescatter.html), and On-line 
Reporting System (password-protected reports 
at http://dpi.state.wi.us/oea/doc/wkce-crt_wsas_ 
trnr_manual. doc) . 

Other student performance indicators that can be 
accessed via WINSS include retention rates, and 
performance on college admissions (ACT) and 
placement (Advanced Placement Program) tests 
(seehttp://data.dpi.state.wi.us/data/performance.a 
sp?fullkey=ZZZZZZZZZZZZ&DN=None-FChosen 
&SN=None-FChosen&TypeCode=08i:ORGLEVEL= 
ST). Detailed information on ACT and AP ex- 
amination data available via WINSS (including 
the source and cautions regarding the use of the 
data) is available online at http://www.dpi.wi.gov/ 
spr/colleg_q&a.html. Additional information on 
educational assessment in the state of Wisconsin 
is available online from the Department of Public 
Instruction’s Office of Educational Accountability 
(OEA, at http://dpi.wi.gov.oea/assessmt.html). 

Graduation and dropout data. Wisconsin collects 
student-level high school completion data, includ- 
ing by credential type (graduates are students who 
received regular diplomas), and student-level drop 
out data. This information is collected annually 
and is stored in Oracle tables at DPI. WINSS pro- 
vides access to this data (e.g., high school comple- 
tion rates, overall and by type of credential, at 
http://data.dpi.state.wi.us/data/graphshell.asp?Gro 
up=AllStudentsFAY&GraphFile=HIGHSCHOOL 
C0MPLETI0N&STYP=9&DETAIL=YES& 
CompareTo=PRIORYEARS&ORGLEVEL=ST& 
FULLKEY=ZZZZZZZZZZZZ&DN=None%20 
Chosen&SN=None%20Chosen&TQSUBJECTS= 
SUMALL&TQSHOW=LICSTAT&RelateToTQS= 
EconomicStatus&DISABILITY=APD&HSC= 
ALL8i:TypeCode=0; drop out rates over time 
at http://data.dpi.state.wi.us/data/graphshell. 
asp?GRAPHFILE=DROPOUTS). Also available 
online is detailed information on high school 
completion credentials data collection and report- 
ing for 2003-04 and beyond (at http://dpi.wi.gov/ 
Ibstat/datahsc.html). 
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Teacher data 

Staff data are collected annually via the Fall Staff 
Report (PI 1202). As described in the Data Defi- 
nitions and Instructions for the 2004/05 school 
year Fall Staff Report (available online at http:// 
dpi.wi.gov/lbstat/doc/1202doc05.doc), informa- 
tion reported in this data collection includes: staff 
employed by reporting agencies as of the third 
Friday in September; staff assignments; gender; 
year of birth; race; highest degree; years of experi- 
ence (within the district, and in total); and salary 
information. Links to a variety of information 
regarding the Fall Staff Report are available on 
the online Data Collection Reporting Schedule (at 
http://dpi.wi.gov/apps/schedule.html). Links to 
information on staff, teacher, and program data 
available at DPI are available at http://dpi.wi.gov/ 
sig/dm-stafifchr.html. Other useful online sources 
for understanding and using teacher data include 
“Understanding Data About Teacher Qualifica- 
tions” (at http://dpi.wi.gov/spr/teach_q&a.html), 
and “Questions to Consider When Using Teacher 
Data” (at http://dpi.wi.gov/spr/teach_use.html). 

Optional perceptions data tools 

District and school users may choose to collect 
and report perceptions data using WINSS tools 
(see http://dpi.wi.gov/sig/improvement/process. 
html). Most of these tools were developed with the 
support of NCREL. Some samples of these surveys 
are provided at http://goal.learningpt.org/winss/ 
sample.htm, http://goal.learningpt.org/winss/ 
scs/sample.htm, and http://goal.learningpt.org/ 
winss/staff/sampques.asp. WINSS also includes an 
optional school improvement planning tool that 
helps local educators identify priorities and develop 
a school improvement plan through discussion and 
analysis of optional and mandated school data on 
WINSS (http://goal.learningpt.org/winss/sip/). 

Linking state data 

Key to the Individual Student Enrollment System 
was the development of the Wisconsin Student 
Number Locator System. The WSN is 



“a unique, unduplicated number . . . intended to be 
the student’s sole identifier throughout his or her 
PK-12 experience. Parents cannot opt their child 
out of being assigned a number in the system. . . . 
Districts use Wisconsin Student Numbers (WSNs) 
instead of names to submit data about student 
educational progress, such as attendance, grade 
level placement, dropout, graduation credentials, 
or acquisition of English proficiency, in addition 
to demographic data necessary to meet mandated 
disaggregation requirements,” (see http://dpi. 
wi.gov/lbstat/datawsn.html). 

The WSN provides “opportunities for the consoli- 
dation of data collections so that more information 
and public reports can be created with less data 
allowing educators to spend more time and money 
on serving students and less on data collection 
than would otherwise be required” (see http://dpi. 
wi.gov/lbstat/wslsdupl.html). Wisconsin also as- 
signs unique teacher identifiers. These are updated 
annually for new teachers and have been in place 
since 2005/06 (see http://www.dataqualitycam- 
paign.org/survey_results/state.cfm?st=Wisconsin). 
Currently records cannot be matched across 
students and teachers. 

Wisconsin has procedures in place to prevent two 
different individuals from receiving the same 
unique ID and to prevent the same student from 
obtaining/receiving a different ID when she/he 
changes districts; Wisconsin also has procedures 
to ensure that a teacher does not have two different 
IDs and that two teachers do not have the same ID 
(see http://www.dataqualitycampaign.org/survey_ 
results/state.cfm?st=Wisconsin). For additional 
information on the state’s education data quality 
assurance procedures, see pages 58-59. 

Longitudinal data decision supports 

Wisconsin is the recipient of a U.S. Department 
of Education Statewide Longitudinal Data System 
grant to “create a longitudinal data decision sup- 
port tool kit that will facilitate knowledge-based 
curriculum, assessment, and school operational 
decisions to improve education effectiveness and 
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Notes 



delivery” (see http://nces.ed.gov/Programs/SLDS/ 
PDF/wisconsinabstract.pdf). Working in col- 
laboration with the Wisconsin Center for Educa- 
tion Research and the Departments of Education 
in Minnesota and Michigan, the Department of 
Public Instruction is working to “develop a com- 
prehensive data system to meet new federal, state, 
and local reporting requirements for schools” (see 
http://dpi.wi.gOv/eis/pdf/dpi2005_154.pdf). Ad- 
ditional information about the proposed program 
of work is available from the U.S. Department of 
Education’s Institute of Education Sciences (see 
http://nces.ed.gov/Programs/SLDS/stateinfo.asp). 



1. As described in the Wisconsin Department 
of Public Instruction’s Data Collection Plan, 
“all data collections which are to be distrib- 
uted to anyone outside of Wisconsin state 
government by or under the sponsorship of 
the Department of Public Instruction must 
be submitted for review and authorization 
prior to planned distribution,” (http://www2. 
dpi.state.wi.us/datacoll/AboutPub.htm). 

From the “Data Collection and Forms” page 
on the DPI web site (http://dpi.state.wi.us/ 
dltcl/dm-datasubmission.html) a variety of 
information about the Plan can be accessed, 
including: a message from the State Superin- 
tendent; background information about the 



publication; help in viewing the Plan; and a 
list of frequently asked questions (see http:// 
www2 . dpi. state.wi.us/datacoll/default . asp) . 
From the latter page it is also possible to 
view the entire list of data collections for the 
DPI, and to search and view collections for a 
particular DPI team/program, by keyword, by 
primary respondent, by month due, by form 
number, or by form type (including survey/ 
questionnaire, sample forms, and miscella- 
neous forms). 

2. The DPI was the “first [Iowa] state agency 
to make use of electronic data reporting”; 
its web-based collections “require on-line 
completion and submittal” and may “include 
database integration whereby submitted data 
is saved to a server database,” (see “Frequently 
Asked Questions” at http://www2.dpi.state. 
wi.us/datacoll/WebHelp/DCP_FAQ.htm and 
“From the State Superintendent” at http:// 
www2.dpi.state.wi.us/datacoll/StSupt.htm). 

3. The mission of the Library and Statistical 
Information Center, “a combined effort of the 
Wisconsin Department of Public Instruction’s 
Professional Library and Statistical Informa- 
tion Center,” is “to provide accurate informa- 
tion about children, schools, school districts, 
public libraries and education” (http://dpi. 
wi.gov/lbstat/). 
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APPENDIX I 

INSTRUCTIONS FOR ACCESSING THE 
STATE DATA INVENTORIES 

Access to the state data inventories will be pro- 
vided through a password-protected web site being 
established at Michigan State University. State 
education agency officials seeking credentials to 
access this web site should contact the Office of the 
John A. Hannah Chair in the College of Education 
at Michigan State University (517 432 0300). 
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